Y7 T5 Token Counter

Here's the problem this node exists for: Flux doesn't read your prompt with one encoder, it reads it with two. CLIP-L takes the first 77 tokens, and the big language model - T5-XXL - gets up to 512 (or 256 on Flux.1 Schnell). Most people feed the same text to both, which means anything past token 77 is T5-only, and anything past the T5 limit never gets read at all. The Y7 T5 Token Counter is the tool that tells you exactly where that cut happens, with the actual T5 tokenizer, before you waste a generation discovering it the hard way.

If you've ever had a long, carefully written Flux prompt come out as if the last three sentences never existed, this is the node that shows you why.

What it shows you

Connect a string and it runs it through the T5 XXL tokenizer and reports:

• Token Count: X / 512 - your raw count against the wall, with a >>N over limit<< marker if you blew past it.
• The last token within the limit and a short window of context leading up to it - so you can see exactly which word gets cut off mid-sentence.
• A numbered token list (up to the limit) and a separate overflow section with the tokens that would have been dropped.
• A Copy Text button so you can grab the diagnostics or the original prompt.

Two on-node toggles do the real work: show_tokens switches the full token dump on and off, and tokens_per_line controls how many tokens per row (4 is the default). There's also a font-size widget for when the dump gets long.

Reading the output takes a second to get used to. Because T5 uses the SentencePiece tokenizer, tokens starting with an underscore (_cat) represent a word boundary - a new word after a space - while tokens without one are subword pieces that continue the previous word. So _ prefix roughly means "space before this piece." The counter also quietly normalizes em/en dashes to plain hyphens, because those can survive prompt-enhancer output as weird characters otherwise.

The inputs and the mechanism

The backend is minimal: one text_in input (marked forceInput, so it's designed to be wired from a prompt node rather than typed into) and a text_out pass-through - the node hands your original string straight through unchanged while showing you the analysis. That pass-through is deliberate: you can drop this node into a prompt chain, read the count, and keep feeding the same text onward without breaking anything.

Under the hood it loads the bundled T5 tokenizer from the pack's text_encoders/t5_tokenizer folder, so it works offline. If those files are ever missing or corrupt it falls back to downloading the google/t5-v1_1-xxl tokenizer from HuggingFace on first use - a small download, and only in that fallback case. One detail the author gets right: T5 has no BOS token but appends an EOS, so the usable budget is actually the limit minus one. 512 means 511 real tokens plus the end-of-sequence token.

Why you'd wire it in

Pair it with any text-generating node - a prompt enhancer, an LLM call, a complex concatenation - and check the count before you hit KSampler. The pack's own prompt-enhancer documentation makes the strategy explicit: front-load high-impact keywords for CLIP's 77 tokens, then let the flowing prose run for T5. This counter is how you verify you haven't let the prose push your keywords past token 77, and whether you're riding right at the 512 limit.

Installing it

Part of ComfyUI-Y7Nodes ("Y7Nodes for ComfyUI") - ComfyUI Manager → Custom Nodes Manager → search Y7Nodes → Install → restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/yushan777/ComfyUI-Y7Nodes
pip install -r requirements.txt

It needs transformers and sentencepiece, both in the pack's requirements. No model files beyond the bundled tokenizer.

Common issues

If the tokenizer silently re-downloads on first run, you're on the fallback path - internet required, happens once, caches locally after. And one honest caveat from the node's own description: it counts tokens as the T5 XXL tokenizer sees them. Whether ComfyUI's actual Flux implementation truncates, splits into chunks, or does something else past the limit is up to that implementation - so treat this as "what the T5 encoder is reading," not necessarily a pixel-perfect mirror of the final conditioning.