Fens Token Counter

If you've ever written a long SD 1.5 or SDXL prompt and gotten output that ignores the back half, that's not your prompt being bad - it's the 77-token wall. CLIP-family encoders truncate at 77 tokens per chunk (154, 231, and so on if you use BREAK), and ComfyUI does the truncation for you, silently. WebUI Forge has shown live token counts for years; in ComfyUI you're usually flying blind. Fens Token Counter is the fix: it asks your actual text encoder "how many tokens is this, and how much of the window did it use?" and prints the answer on five outputs.

The "ask your actual text encoder" part is the whole point. The node doesn't guess from a word count or a regex - it hands your text to the real clip.tokenize() pipeline via the CLIP object you wire in, so the count reflects the exact tokenizer your model uses: one stream for SD 1.5, the l+g pair for SDXL and its derivatives, t5xxl for FLUX, qwen3 for Anima. The numbers change when you swap checkpoints. That's correct behavior, not a bug - the tokenizer genuinely changes.

How it works

The node tokenizes your text per stream, counting real (non-padding) tokens. The count_strategy combo decides how to aggregate across tokenizer branches: max_stream (the default) reports the largest single branch, which is the honest answer for SDXL where both CLIP-L and CLIP-G encode the full prompt; sum_streams adds all branches together. On single-encoder models like FLUX or Anima the two are identical, so you can stop thinking about it there.

It also speaks prompt syntax, which is rarer than it should be. (word:1.3) weight syntax doesn't add tokens - it changes embedding strength - and the counter knows that. BREAK is split per segment and padded exactly the way ComfyUI's conditioning pipeline does it, and escaped \( \) won't fool the parser into miscounting.

The inputs and outputs that matter

Wire the CLIP output from your checkpoint or CLIPLoader - the same one feeding your CLIP Text Encode - then type into the text widget or connect a string. Of the five outputs, the two you'll actually read are Total Tokens (what you typed, minus padding) and Context Limit Tokens (the padded window: 77/154/231 for CLIP-style encoders; just a minimum padding floor for unbounded T5/Qwen3 encoders, not a hard ceiling, per the author's own tooltip). Chunk Count tells you when your prompt spilled into a second 77-token chunk, Details is a human-readable summary, and Prompt Echo hands the input string back out.

The trick everyone misses: route Prompt Echo into your CLIP Text Encode text input. Now the counter sits inline and can never drift from what's actually being encoded. show_token_breakdown (an advanced toggle) appends a per-token dump - id, decoded text, weight, word id - to Details. It's a debugger's toy, but occasionally the only way to see why a tokenizer mangled your prompt.

Installing it

ComfyUI Manager → search "Fens-Simple-Nodes" → install → restart, or clone it manually:

cd ComfyUI/custom_nodes
git clone https://github.com/Taithrah/ComfyUI_Fens_Simple_Nodes

Then restart ComfyUI. No model downloads, no extra Python dependencies - it runs entirely on ComfyUI's bundled tokenizers, torch, and PyYAML. One real catch: the code targets ComfyUI's newer V3 node API (comfy_api.latest), so on a stale ComfyUI build the node won't register at all. Update ComfyUI before you file a "missing node" bug.

Troubleshooting

The common failure is forgetting to connect the CLIP input - every output zeroes out and Details shows a warning. If Total Tokens is larger than Context Limit Tokens, your prompt is being chunked (or worse, cut), and that's the 77-token wall showing up in real time. And on FLUX or Anima the wall is gone, but the count still matters: past roughly 75–100 effective tokens even unbounded LLM encoders start to drift. Knowing the number is the first step to trimming the prompt.