D2 Token Counter

If you've ever pasted a prompt into Stable Diffusion and watched it silently truncate past 75 tokens (or 77, or 225, depending on your CLIP setup), you know why a real token counter is useful. D2 Token Counter is that: it runs your prompt through the actual CLIP tokenizer and reports the token count - not "number of words," which is a lie - plus the tokenized result. It's the difference between a prompt that fits and one that's quietly lost its last three tags.

The pack's own D2 Prompt node shows a token count inline, but it hardcodes the ViT-L/14 tokenizer. This node is the one you reach for when you want a specific CLIP model or you need the count as a data output you can branch on.

The inputs and outputs

• text - the prompt to count.
• clip_name - which OpenAI CLIP tokenizer to use: ViT-L/14 (default), ViT-B/32, or ViT-B/16. For SD1.5/SDXL text encoding, ViT-L/14 is the one that matches what the model actually uses.

Outputs:

• token_count - the INT count.
• tokenized_result - the tokenized output as a string (handy for seeing exactly where a tokenizer splits a word, which is where surprises live).

How it works

It loads the tokenizer through HuggingFace's transformers (CLIPTokenizer.from_pretrained), which means two real-world details you should know: the first run needs a network connection to download the tokenizer files, and the pack requires transformers to be present in your ComfyUI environment. ComfyUI almost always has it, but if you're on a stripped-down install, that's the dependency to watch. If loading fails, the node falls back to the ViT-L/14 tokenizer rather than dying, so the failure mode is "wrong tokenizer," not "crash."

Why does this matter? CLIP's BPE tokenizer splits subwords, not words: masterpiece might be one token while highly_detailed splits into a few. Counting spaces badly undercounts long compound tags, and that's precisely where truncation bites. A real tokenizer count tells you the truth.

Installing

Part of the D2-nodes-ComfyUI pack - install the pack, not the node.

• ComfyUI Manager → search "D2-nodes-ComfyUI" → Install → restart.
• Or cd ComfyUI/custom_nodes && git clone https://github.com/da2el-ai/D2-nodes-ComfyUI and restart.

The pack's own requirements are just piexif and charset-normalizer, so if transformers isn't in your environment, add it (pip install transformers in your ComfyUI venv). No model downloads. Pack gotcha: v32.0.0+ needs the ComfyUI V3 schema; older ComfyUI → pre-32.0.0 release.

Where people get burned

The big one: the tokenizer choice must match your model. Counting with ViT-B/32 and assuming the result applies to an SDXL prompt is wrong - SDXL's text encoder uses the large patch-14 tokenizer, so leave clip_name on the default unless you know otherwise. Also, remember this counts raw tokens; the 75-token limit you're checking against applies per CLIP-encoded chunk, and ComfyUI may handle overflows by chunking rather than truncating. The counter tells you how many tokens you have; whether that's a problem depends on your encoder setup.