LLM Sampler

You already have a text encoder, so why do you need a whole LLM node in ComfyUI? Because prompt generation is the one job your CLIP encoder can't do. LLMSampler is the pack's general-purpose chat node: give it any GGUF model and it answers. No images required - though it'll happily use a LLaVa model as its brain if that's what you loaded. It's the node you reach for when you want a local model to write prompts, expand ideas, or just chat while you build a workflow.

What it is

LLMSampler sits in the pack's LLM family alongside the prompt-generation nodes. The difference: those are wired to produce specific outputs (prompts, keywords, structured values), while this one is the plain conversational sampler. The README is explicit - "you can chat with any LLM in gguf format, you can use LLava models as an LLM also." If you want to ask a Qwen model "what would a good prompt for this be?" and get a thoughtful reply, this is the node.

The inputs that matter

• system_msg - default "You are an assistant who perfectly describes images." Change it to fit the job. For prompt work, something like "You are an expert prompt writer for SDXL" does more for your output than any sampling tweak.
• prompt - your question or instruction. Multiline.
• model - CUSTOM: the GGUF model from the pack's loader node. LLM GGUFs and LLaVa GGUFs both work.
• max_tokens - cap on the reply, default 512.
• temperature - default 0.2. Your creativity dial; low for consistent, up for variety.
• top_p / top_k / frequency_penalty / presence_penalty / repeat_penalty / seed - the standard sampling panel. Defaults are reasonable; raise repeat_penalty (1.1) when the model loops.

Output is a single STRING.

How it works

Straight llama.cpp chat inference: system message, your prompt, token-by-token generation with the sampling parameters applied at decode time. Because the model comes in through a loader node, context size, GPU offload, and the GGUF/projector pairing are all configured upstream - swap the loaded model and this node just follows along.

Install

Pack-level install:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes

Restart ComfyUI (or Manager → "VLM Nodes"). Python 3.9+ and the llama-cpp-python stack. Drop your GGUF into models/LLavacheckpoints; if it's a LLaVa model, its matching mmproj clip file goes alongside it - same HF repo, always.

Troubleshooting

The usual llama-cpp-python suspects: on Windows, the "invalid wheel filename" install error and the missing llama.dll import failure people hit on startup. Both resolve by reinstalling llama-cpp-python with the right build args for your backend (CUDA, Metal, ROCm) - the README links the abetlen docs for exactly this. If the model runs on CPU, same cause. If answers repeat, raise repeat_penalty. If the model seems smart but the output is garbage anyway, check that your GGUF and projector match.