LLaVA Sampler (Advanced)

The simple LLaVa sampler gets you a caption. This one gets you control. LLavaSamplerAdvanced is the full-instrumented version of the pack's LLaVa node: same idea - a vision-language model describing your image - but with every sampling knob exposed so you can actually steer how the model answers instead of just taking whatever it gives you.

What it is

Where LLavaSamplerSimple gives you image, prompt, model, and temperature, this node adds max_tokens, top_p, top_k, frequency and presence penalties, repeat_penalty, and a seed - plus a system_msg you can set. That matters more than it sounds. Captioning for prompt generation is one thing; asking a VLM to classify an image, extract attributes, or answer the same question repeatedly for batch work is another, and that's where penalties and a fixed seed earn their keep.

The inputs that matter

• image - the picture under discussion.
• model - CUSTOM, the GGUF LLaVa model from the pack's loader node. Same deal as the simple version.
• system_msg - default "You are an assistant who perfectly describes images." This is your cheapest lever for changing behavior: swap it for "You are a strict classifier" or "List only tags" and the model follows the role.
• prompt - your actual question or instruction.
• temperature - default 0.1. The main creativity knob.
• top_p / top_k - nucleus and top-k sampling. Defaults (0.95 / 40) are sensible; drop them for more deterministic output.
• repeat_penalty - default 1.1. Crank it if the model starts looping or repeating itself, which smaller GGUFs love to do.
• seed - default 42. Set it to make runs reproducible; that's invaluable when you're debugging a prompt.

Everything else - max_tokens, frequency_penalty, presence_penalty - behaves like you'd expect from any LLM sampler. Output is a single STRING.

How it works

Mechanically it's llama.cpp inference over your loaded GGUF: image in, prompt plus system message, token-by-token generation with the sampling parameters applied at decode time. The node itself doesn't load the model - the loader does - so context size and GPU offload stay where you configured them. The advanced part is purely what happens during generation.

Install

Pack-level install, nothing node-specific:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes

Restart ComfyUI (or Manager → search "VLM Nodes"). Python 3.9+, and you need your LLaVa GGUF plus its matching mmproj clip projector in models/LLavacheckpoints - from the same HF repo, always. The llama.cpp stack (llama-cpp-python) is the dependency that actually causes setup pain.

Troubleshooting

The two gotchas are the pack's universal ones: llama-cpp-python install failures on Windows (the "invalid wheel filename" and missing llama.dll import errors people report) - fix by rebuilding it for your backend - and mismatched GGUF/projector pairs, which produce confident-sounding nonsense. When output repeats itself, raise repeat_penalty before you touch anything else. And remember: if you don't need the dials, the simple node is right there.