Nodes/ComfyDL/Vocab Decode
ComfyUI Node

Vocab Decode

Numbers back to words — the node that makes model output readable

By Cynthia-lxx·Created 2 months ago·Updated 3 days ago· 6
Vocab Decode
  • vocab
  • indices
  • tokens_str

Your model just handed you a tensor of token indices and you have no idea what it said. Vocab Decode is how you find out. Give it a vocabulary and an index tensor, and it looks each index up in the reverse table, returning the tokens as a comma-separated string. It's the d2l Vocab.to_tokens as a node, and it's the "read the output" half of the encode/decode pair - the thing you wire at the end of an RNN language model so the generated sentence comes out as words instead of a wall of integers.

The mechanism is a reverse dictionary lookup, one index at a time, and it's about as forgiving as it can be: indices that fall outside the vocab - say, a model that hallucinated a token number beyond your table - get mapped to <unk> instead of crashing. Which is exactly what you want at the end of a generative pipeline, because the alternative is the whole queue dying one token short of the finish line.

Inputs

  • vocab - the cdlVocab object from Vocab Build, the same one you encoded with. Decoding with a different vocab than you encoded with is the classic way to get gibberish.
  • indices - a cdlTensor of integer indices (long dtype, as produced by Vocab Encode). A single index or a whole sequence both work.

Output

One output: tokens_str, a STRING of comma-separated tokens. If you feed it a full sentence's worth of indices you get the whole sentence back in one line. It's a string, so it lands nicely in text/display nodes and lets you actually read what the model generated.

Installing it

Part of ComfyDL. ComfyUI Manager, search "ComfyDL". Or:

cd ComfyUI/custom_nodes
git clone https://github.com/Cynthia-lxx/ComfyDL
pip install -r ./ComfyDL/requirements.txt

Restart ComfyUI. Pack-wide install is just matplotlib; nothing to download.

Gotchas

Match your vocab, always. The single most common mistake is building a vocab, encoding with it, and then decoding with a rebuilt (differently-sorted, differently-sized) vocab - the indices shift and you get a line of <unk>s and wrong words. This node is also your best debugging tool for the whole NLP chain: wire Vocab Decode at the end of your model output and at the input (via round-trip encode→decode) and compare. If encode gives you back different tokens than you started with, your vocab doesn't contain your text - and the <unk> collapse is happening before the model ever sees anything. Decode is the read-out that makes that visible. One format note: the output is comma-separated, which reads a little oddly for a sentence, but it's the pack's wire format and it flows cleanly to the display nodes. Young pack, no community threads yet - but this is a simple reverse lookup, so when it "misbehaves," the bug is almost always a vocab mismatch upstream.

Categoryd2l/NLP Utils

Inputs (2)

NameTypeDefaultDescription
vocabcdlVocab
indicesTENSOR

Outputs (1)

NameTypeDescription
tokens_strSTRING