LLM_Sample

Sampling is the exact moment a language model "decides" what to say next. LLM_Sample is that decision, exposed as a node: after the model's context has been primed, it returns a single INT - the token ID the model picked as the next one. It's one of the lowest-level nodes in this pack, and it's where you'd go if you want to watch generation happen one token at a time.

To be clear about what this isn't: LLM_Sample does not return readable text. It returns one integer, a token ID, chosen from the model's vocabulary according to its probabilities. To see what it picked, you run that ID through LLM_Detokenize. To prime the model's context so it has something to sample from, you run LLM_Tokenize into LLM_Eval first. Sample is a single piece in that loop, not a complete generator.

Inputs and output

• LLM - the loaded model. The node samples from whatever's currently in that model's context.
• top_k (40) - limit candidates to the top 40 tokens by probability.
• top_p (0.95) - nucleus threshold: only sample among tokens whose cumulative probability reaches 0.95.
• temp (0.8) - temperature. Below 1 sharpens the distribution toward the most likely token; above 1 flattens it into chaos.
• repeat_penalty (1.1) - suppresses tokens that have appeared recently, to discourage loops.

Note the slightly different naming vs. the Call LLM nodes - temp here, temperature there. Same dial, shorter label. All four are the standard llama.cpp sampling parameters, so the tuning advice is the same: lower temperature for deterministic output, raise repeat_penalty in small steps if the model loops, leave top-k/top-p alone until you have a reason.

The output is INT - a single token ID. Not a list, one number.

When you'd reach for it

Honestly? This is a tinkering node. If you're building the manual inference loop (LLM_Eval to set context, LLM_Sample to pick the next token, LLM_Detokenize to read it), this is the middle of it. It's also the node that makes the "one token at a time" nature of LLMs visceral: run it, get 847, detokenize, get "the", run it again, get 2031, detokenize, get " cat". You'll never again wonder why text generation feels incremental.

It's not for production workflows. Call LLM nodes generate full responses and return strings, which is what you actually want 99% of the time. LLM_Sample is for the lab, and - like LLM_Eval - it runs into the pack's no-loopback limitation if you try to build a real iterative generator out of it. Single-step sampling works fine; unbounded loops don't.

Install

Shared pack setup: ComfyUI Manager search "ComfyUI-Llama", or:

cd ComfyUI/custom_nodes
git clone https://github.com/daniel-lewis-ab/ComfyUI-Llama

with pip install llama-cpp-python on manual installs. GGUF models into ComfyUI/custom_nodes/ComfyUI-Llama/models, restart, then Ctrl+F5. Nodes live under the LLM menu.

Verdict: LLM_Sample is the pack's most exposed nerve - the actual moment of generation, reduced to one number. Great for understanding, rarely needed for shipping.