Clip Tokens Encode (Shinsplat)

Normal ComfyUI prompt weighting works at the word level: (unrelenting:1.3) boosts the whole token sequence for "unrelenting" equally. But words get split into tokens - "unrelenting" becomes un, rel, enting - and sometimes you want one of those pieces weighted differently from the others. That's what this node does, and it's the missing half of the pack's token pipeline: Text To Tokens (Shinsplat) produces the token data, and Clip Tokens Encode turns it into actual conditioning. The two work as a pair, with Tensor Toys as the optional controller on top.

How it works

You feed it a clip plus the tokens_ string that came out of Text To Tokens - a Python-dictionary-formatted list where each element describes one token: its numeric id, weight, the original word fragment, and an index. Instead of handing your prompt text to the tokenizer, this node hands the tokenizer's output back in, letting the weights ride through to the tensors. The README's example is exactly the "unrelenting" case: three tokens, each carrying its own weight key, each independently reweightable. You can edit those weights by hand in the text, or (better) pipe the tokens_ output through a Python node or the Tensor Toys controller to alter them programmatically.

There's an optional control_ input, a string expected to be a Python dictionary or the output of Tensor Toys, which adjusts how the weights get prepared before encoding. Leave it empty if you just want straight token encoding.

The inputs and outputs

• clip - the CLIP model (the pack's Text To Tokens needs the same clip wired in, so the token ids match).
• tokens_ - the structured token string from Text To Tokens.
• empty_padding - none or empty_prompt, controlling the padding stream.
• control_ - optional controller dictionary / Tensor Toys output.

Outputs: CONDITIONING for the sampler, and _tokens - the actual token tensors as a dictionary-formatted string, so you can inspect what's really heading into the KSampler.

How to install it

Ships in ComfyUI-Shinsplat:

cd ComfyUI/custom_nodes
git clone https://github.com/Shinsplat/ComfyUI-Shinsplat

or via ComfyUI Manager, then restart. No requirements file, no models to download.

Common issues

• "tokens_ isn't valid" - it must come from the matching Text To Tokens node (or be hand-formatted to its spec). A raw prompt string won't parse.
• Weights seem to do nothing - token-level reweighting is subtle; start with something obvious like pushing one token to 0.0 or 2.0 and watch what breaks before tuning finely.
• Token ids don't match your model - the tokenizer output depends on the clip you wired in. If you swap models, regenerate the tokens through Text To Tokens rather than reusing the old string.