Y7 CLIP Token Counter

CLIP doesn't count tokens the way you do. A "word" to you is often two or three tokens to it, which is why your carefully-written prompt is silently getting chopped off at the boundary nobody told you about. The Y7 CLIP Token Counter exists to show you exactly where that boundary is - and what's falling off the edge.

It's a debug/utility node from the Y7Nodes pack. If you've ever fed a long prompt into an SD1.5/SDXL/Flux setup and wondered whether the tail end ever made it into the conditioning, this is the node that answers you.

The 77-token thing, briefly

The CLIP text encoder that SD1.5, SDXL, and Flux's CLIP-L half all share truncates at 77 tokens, including two special tokens: <|startoftext|> (BOS) at the front and <|endoftext|> (EOS) at the end. So really you have 75 tokens of usable space. ComfyUI gets around this for some models by splitting long prompts into 77-token chunks and batching them - but how a model handles overflow depends entirely on the implementation. Some truncate, some chunk, some do something weirder. The only way to know what's happening with your prompt is to look.

This node uses the actual CLIPTokenizer from HuggingFace transformers, so what it reports is what the encoder would see - not a character count, not a word count.

What it shows

On the node you get the token count, a clear yes/no on whether the input exceeds the 77-token limit, the final token that made it into the range (with surrounding context so you can see what got cut mid-phrase), and a full breakdown of tokens within the limit plus any overflow. A </w> marker shows where a space followed a word in the original text - useful for spotting where the tokenizer decided to split things.

Inputs and outputs are mercifully minimal:

• text_in - the string to analyze (optional input, defaults to empty).
• text_out - a pass-through of the same text, so the node can sit inline in your prompt chain without interrupting it.

There are a few frontend widgets beyond the schema that matter: show_tokens toggles the tokenized view (needs a re-run), tokens_per_line controls formatting, font_size for readability, and there's a Copy Text button - genuinely handy when you want to paste the token analysis somewhere to stare at it.

When you'd actually use it

Three cases. Debugging why a prompt's later keywords have no visible effect (they got truncated). Verifying your chunking assumption when you switch models. And, honestly, settling arguments - "is this 75 tokens or 120?" is a much better question answered by the tokenizer than by counting spaces.

One honest note from the node docs: longer prompts are supported by ComfyUI, but how overflow is handled is the model's business, not the token counter's. This node tells you what the tokenizer sees; it doesn't tell you what the diffusion model does with the overflow. That's a genuine limitation worth keeping in mind - it's a diagnostic, not a crystal ball.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/yushan777/ComfyUI-Y7Nodes
cd ComfyUI-Y7Nodes
pip install -r requirements.txt

Restart ComfyUI (or install via Manager → search "Y7Nodes"). The transformers dependency the pack pulls in is what does the tokenizing.

Verdict

It's a niche tool - most people never need to see their token stream - but it's the right niche tool: it uses the real tokenizer, it shows you the actual cutoff, and the pass-through means it never breaks your graph to install. If you write long prompts and something isn't landing, this is the fastest way to find out why.