[llama.cpp] Model Profile

Here's a thing nobody tells you when you first run a vision model locally: the model doesn't just need its weights and a projector. It needs a chat handler - the code that knows how to wrap your prompt and your image into the exact format that model was trained on. Get it wrong and the model technically runs, produces fluent text, and completely ignores the picture.

This node is the polite interface over that. You pick a family name, and it sets the handler plus sensible sampling defaults for that family, bundling both into one typed connection.

How it works

The profile combo is populated from JSON preset files the pack ships (presets/model/*.json), so the list you see is whatever is installed: General, General Vision, Gemma 4, Gemma 4 Vision, Qwen 3 VL, Qwen 3.5+ Thinking, Qwen 3.5+ Non-thinking, Qwen 3.8 Thinking, Qwen 3.8 Non-Thinking, Muse Glimmer, plus Custom.

Each profile carries three things: a handler name (auto, generic, gemma4, qwen3_vl, qwen25_vl, qwen3_asr, qwen35), a recommended reasoning mode (auto, off, on) and a full set of sampling defaults. The reasoning hint is quietly the most valuable part - the non-thinking Qwen profiles come with temperature 0.7 / top_p 0.8 / top_k 20 / presence_penalty 1.5, the thinking ones with temperature 1.0, which is roughly the difference between a captioner and a rambler.

Choose Custom and the node ignores the preset entirely and uses its own widget values - temperature, top_p, top_k, min_p, repeat_penalty, presence_penalty - with handler auto.

The output is a typed OLLAMA_IMAGE_LIST_LLAMA_CPP_MODEL_PROFILE cable. Connect it to Create Runtime Session and it applies for the whole session. Connect it to Generate or Decide instead and it's a per-request override - with one caveat straight from the author's tooltip: the session handler is selected at creation and is not changed, so a per-request profile can change sampling but not swap the handler under a live server.

What to actually set

For most people: pick the profile that matches the model you downloaded and touch nothing else. General for a text model, General Vision for an unknown vision model, the specific family profile when it exists. That's the entire intended workflow, and it's why the node looks intimidating at first and then barely gets looked at again.

Then reach for the sampling widgets only when you have a symptom. Repetition loops want repeat_penalty nudged up. Captions that wander into editorialising want lower temperature. Creative writing for a video prompt wants the opposite. And if you're using one node to serve two very different jobs in the same graph - captioning captions here, prose there - two Model Profile nodes is the clean way to keep them apart rather than one node switching values per run. These are Advanced inputs in the pack, so remember to expand the node in the UI before concluding the fields aren't there.

Where people get burned

Connected profile overrides your widgets. If you wired a Model Profile into a Generate node and then edited the node's own sampling values, your edits did nothing - that's the override doing its job. The pack does grey out overridden widgets in recent ComfyUI builds, but muscle memory from older workflows trips people up.

A handler that doesn't support the modality. Vision, audio and video support depends on the model and the handler. qwen3_asr exists for audio-transcription-shaped models; a generic handler on a model with no audio tower will take the audio input and do nothing useful with it. If media seems ignored, suspect the handler name before suspecting the mmproj.

Profiles are typed cables, not numbers. You can't feed a float into it. Custom exists precisely for the case where you just want to type numbers, and it's the honest choice if you don't know which family your GGUF is.

Install

Manager → search llama multimodal → ComfyUI-llama-multimodal. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

ComfyUI 0.19.3+. No model is required by this node itself; it becomes useful the moment you connect a Runtime or Native Session, and those want the llama runtime (Settings → llama-multimodal → Download llama.cpp), a GGUF plus matching mmproj in ComfyUI/models/LLM, and - for Native sessions - the optional multimodal llama-cpp-python fork.

Because the presets are plain JSON on disk, adding your own profile is a copy-and-edit job rather than a code change. Give it a name, a handler, and your numbers; it'll appear in the combo after a restart.