Llama.cpp Generate (Multimodal)

If the Ollama node is the "talk to a server" path, Llama.cpp Generate (Multimodal) is the "run it in my own process" path - and it's the one the community has been asking for since the KB's LLM-in-ComfyUI doc tracked the standing want: "Would be nice if we could use any GGUF llm model without ollama tho." This node loads one GGUF directly inside ComfyUI through llama-cpp-python, does exactly one stateless multimodal chat completion, then closes and releases the model immediately. No model cache, no retained context, no separate server process.

The mechanism is worth being precise about, because it explains both the appeal and the gotchas. Any request with IMAGE, AUDIO, or VIDEO needs a matching mmproj GGUF (the multimodal projector for that exact model family) - without it, media is rejected. The handler input picks the chat template: auto uses model metadata and the fork's generic MTMD path, and you can pin gemma4, qwen3_vl, qwen25_vl, or qwen3_asr when a template requires it. thinking is an explicit boolean request - it doesn't infer a model default, and it can't turn an Instruct-only checkpoint into a thinking checkpoint. All your media becomes one user message (order always IMAGE, then AUDIO, then VIDEO), and it all lands in one completion call.

Model discovery is tidy: the node scans .gguf files recursively from every ComfyUI model path whose category is LLM (including extra_model_paths.yaml entries) plus models/LLM as a fallback. Both model_path and mmproj_path see the full inventory; the projector list just puts names containing mmproj first for convenience.

The inputs you'll touch most: model_path and mmproj_path (pick the GGUF pair), system and prompt, n_ctx (default 8192, and vision models eat context fast), max_tokens, gpu_layers (all/auto/CPU), and the sampling params with defaults tuned low (temperature 0.2, top_p 0.95). The advanced section has real teeth, so read the tooltips: image_max_tokens (default 1120) sets the per-image token ceiling, and n_batch/n_ubatch must be large enough to cover it or you'll fail before the model even loads. Two optional typed inputs clean things up: a Sampling Preset overrides the five sampling floats, and a Gemma 4 Runtime Preset overrides the batch/token overrides. Outputs are response, thinking, raw JSON, metrics, and a media diagnostics receipt you can expand with the pack's diagnostics node.

Now the honest part: the dependency. llama-cpp-python is deliberately not a package requirement, because this targets JamePeng's multimodal fork and a usable wheel must match your OS, CPU architecture, the exact Python ABI of the interpreter that starts ComfyUI, and your native backend (CUDA, ROCm, Vulkan, Metal). The CUDA tag on your PyTorch build does not select the wheel - they're separate native runtimes. Install it into ComfyUI's actual Python environment, not a random system Python:

# Windows portable: use the embedded interpreter, e.g.
.\python_embeded\python.exe -m pip install llama-cpp-python   # a JamePeng fork wheel

See the JamePeng releases page and the ComfyUI-ThinkingLLM install notes linked in the error message. VIDEO input additionally requires a wheel built with MTMD_VIDEO support (no separate FFmpeg needed - it uses llama.cpp's native libmtmd helper). Restart ComfyUI after installing the wheel. If the wheel is missing, everything else in the pack still loads; only this node errors, with the install guide in the message.

Getting the pack itself is the usual:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-Ollama-ImageList

One heads-up for beginners: this detailed node lives under Ollama / llama_cpp / legacy and is marked V3 development-only, so normal ComfyUI sessions hide it from search and the add-node menu - enable developer mode to see it. The pack's compact "Llama.cpp Generate" is the one meant for everyday use. If you're new to GGUF, the KB's format panel is the right primer: Q8 is basically fp16 at half the size, and the projector is not optional for vision. When the backend rejects something, you get its real error - this node never resizes, drops, or silently splits media to force a request through.