Llama.cpp Generate

"Llama.cpp Generate" is the everyday workhorse of this pack's llama.cpp half, and its design philosophy is right there in the category: compact. Where the legacy detailed node drowns you in thirty-plus advanced widgets, this one keeps only the things you actually change per request visible - model, prompts, context, token budget, media - and pushes the stable tuning knobs behind typed sockets you connect or don't. It runs one multimodal llama.cpp completion per execution, same stateless GGUF-in-process model as its detailed sibling, without the clutter.

The required inputs mirror the detailed node: model_path and mmproj_path from ComfyUI's LLM model folders (the projector is mandatory whenever you connect images, audio, or video), system and prompt, n_ctx (8192 default), max_tokens, seed, and stop. The one required socket is model_profile, which must come from the pack's Llama.cpp Model Profile node - that's how it gets its handler and sampling defaults, and it's non-negotiable, so the wiring is always explicit. Two visible request budgets are worth understanding: image_max_tokens and image_min_tokens, both defaulting to 0. A 0 means "leave the projector/handler default alone"; a positive value is an explicit override. The tooltip on image_min_tokens is the practical one - Qwen-VL grounding tasks often need 1024.

The optional sockets are where the flexibility lives:

• hardware_profile - from Llama.cpp Hardware Runtime Profile. Disconnected means GPU Full Offload (all layers, n_batch 512, main GPU 0, auto threads/flash attention, mmap on).
• reasoning - from Llama.cpp Thinking / Reasoning Config. Disconnected means the model's default reasoning behavior.
• speculative - from either the Compact N-gram or the Experimental Native Speculative Config node. They share one socket; plug in one, not both.

The images, audio, and video inputs accept single items, batches, lists, or nested lists and flatten deterministically into a single completion - the same one-call list semantics as the Ollama node. Media order is always images, then audio, then video, and scalar inputs like prompts must resolve to exactly one value.

Outputs are response, thinking, raw JSON, metrics, and media diagnostics - the last being a typed receipt you can hand to the Media Diagnostics node if you want capability flags and counts without squinting at JSON.

Setup is shared with the whole pack:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-Ollama-ImageList

...restart ComfyUI, and make sure the JamePeng llama-cpp-python fork wheel is installed in ComfyUI's Python environment (see the detailed node's page for the install saga - the compact node has the same dependency). It also needs ComfyUI 0.19.3+ for the V3 API.

My honest take: if you're going to use llama.cpp in this pack at all, use this node, not the legacy one. Profiles give you per-model presets you can reuse across workflows, and when something misbehaves you have three typed inputs to suspect instead of thirty widgets to scroll. The learning curve is "make a Model Profile, connect it, done" - everything else is optional by design.