Prompt Minimizer And Splitter Node (zer0)

If you've ever pasted a prompt into ComfyUI and wondered whether you're blowing the token budget, or watched a CLIP model mangle the last half of a long tag list, this node is the answer. The Prompt Minimizer And Splitter Node (zer0) does three things in one pass: strips the junk whitespace from your prompt, counts the tokens before and after, and optionally re-splits the prompt into token-sized chunks with a separator between them.

The author is upfront that it's a mashup - "based heavily on the Tiktoken Tokenizer and String Cleaning nodes" from MNeMiC Nodes, merged into one node because nobody had exactly this combination. That's the pack's whole philosophy, honestly: personal utilities that plug a gap you keep hitting.

How it works

It uses OpenAI's tiktoken library to do the counting. On the input_string you feed it, it splits on commas, strips leading/trailing whitespace from every tag, and rejoins with plain commas. Then it encodes each tag and chunks them greedily until the running total would cross token_count (default 75), starting a new chunk and joining chunks with your separator (default " BREAK "). The numbers on the outputs are real, from the tokenizer, not an estimate.

The inputs that matter:

• input_string - your raw prompt, multiline. Paste the whole sloppy thing in.
• encoding_type - which tiktoken encoding to count with: gpt-4o, gpt-4, gpt-3.5-turbo, or the base encodings o200k_base, cl100k_base, p50k_base, r50k_base. Default is gpt-4o. For prompt-token purposes they're all close enough; cl100k_base is the classic SD-era default if you want to match what other tools report.
• token_count - the maximum token length before a chunk breaks. Default 75, which is the old CLIP chunk size, and it's a sane default for SD 1.5/SDXL-lineage models.
• separator - what gets inserted between chunks. BREAK is the default and the classic choice.

Four outputs: Optimized Prompt (whitespace stripped, not split), Chunked Prompt (stripped and split), Token Count (Unoptimized) and Token Count (Optimized) - both INTs. Because this is an output node, the counts display right on the node, which is the part I actually like: you finally get a number for how close you are to the limit.

Where it helps, and where it's theater

The honest caveat: 75-token chunking with BREAK is a CLIP-era technique. It does real work on the SDXL lineage - Illustrious, Pony, NoobAI and friends still have the 77-token chunk boundary, so breaking your prompt across it keeps late tags from bleeding together. On LLM-encoded models (Flux 2 Klein, Z-Image, Anima) there's no such boundary, and the chunked output is mostly pointless. But the token counts stay useful everywhere: those models have a real attention cap around 75–100 effective tokens, and having a live counter for how far you've overshot it is genuinely worth the node.

One quirk to know: the chunker uses a strict < against token_count, so a single tag that lands exactly on the limit gets pushed into the next chunk. And a tag longer than the limit alone will still occupy its own chunk. That's splitting-by-tags behaving as designed, not a bug to fight.

Install

This is the one node in the pack with a real dependency, tiktoken, and the import lives at the top of the module - so if it's missing, the entire pack fails to load, not just this node.

cd ComfyUI/custom_nodes
git clone https://github.com/zer0thgear/zer0-comfy-utils
cd zer0-comfy-utils
pip install -r requirements.txt

Restart and find it under text utility. The easy route is ComfyUI Manager → Install Custom Nodes → search "zer0 Comfy Utilities" - Manager installs tiktoken for you. No model downloads anywhere in the pack; this is pure text math.