[llama.cpp] Create Native Session

Every other llama.cpp node in this pack takes a session. This is where the session comes from, and it's the node that decides how much VRAM you're holding and for how long.

If you've been running an LLM in ComfyUI by talking to an external Ollama server, this is the other pattern: the model loads inside the ComfyUI process, once, and stays resident while the rest of the graph uses it. That's the standing community wish the KB tracks - "use any GGUF llm model without ollama" - and this node is that, done locally with llama.cpp.

Why a session node exists at all

Because reloading a 12B GGUF from disk per node call is miserable, and because in-graph LLM work is usually several calls in a row: caption the image, then rewrite the caption for the model you're actually sampling with, then translate it. The pack's own description tells you the intended shape - put this before your loop, carry session through to the end. One load, many requests.

Inputs that matter

model_path is a combo populated from ComfyUI/models/LLM. If it reads [no GGUF models found], you haven't put a file there yet. The pack registers that folder itself (it also picks up any llm folder you've configured), and only .gguf files show up. Drop one in, refresh the node list.

mmproj_path is the multimodal projector - the second GGUF that gives a vision-capable model eyes. Text-only work: leave it on [none], and the pack skips loading a projector entirely rather than pretending. Vision uses both files, so they have to match: a projector from a different model family will load and then produce garbage, which is worse than an error.

custom_handler picks the chat/ingest handler. auto takes it from the Model Profile; the manual options are generic, gemma4, qwen3_vl, qwen25_vl, qwen3_asr and qwen35. There's an audio-specific handler in that list (qwen3_asr) - audio and video support is genuinely model-dependent, so check what your GGUF can do before wiring a waveform into it.

n_ctx is the context window, default 8192, up to 1048576 in steps of 512. Media eats this fast: an image is a few hundred to a few thousand tokens depending on the model and your prefill profile. Long video clips plus a long prompt is how people hit the wall.

verbose is an advanced boolean that dumps llama.cpp chatter to the console. Turn it on for exactly one debugging run.

Everything else is optional and typed: model_profile, custom_chat_template (a Jinja template; leave it disconnected and the GGUF metadata template is used), hardware_profile, reasoning, speculative, prefill_profile. Each is its own node in this pack that outputs a typed blob, and connecting them is equivalent to passing the corresponding llama.cpp flags. You don't need any of them to get started.

One output: session. It goes into Generate, Generate (Prompt Sequential), Generate (Media Sequential), Decide, or Unload Session.

Loading, unlucky timing, and unloading

The node is marked non-idempotent and fingerprints itself with the wall clock, so ComfyUI re-runs it every queue rather than caching a stale session. That's what you want after changing n_ctx or swapping the GGUF.

The flip side is that a session left open holds VRAM. The pack tracks sessions and has a cleanup hook that fires at prompt end, so an interrupted or cancelled workflow closes what it opened instead of leaking memory into the next prompt. You still want Unload Session (or session_unload on your last generate) when you're deliberately alternating between a vision pass and your diffusion model on one card.

Where people get burned

The pack does not bundle llama-cpp-python. It targets JamePeng's multimodal fork specifically - the one that exposes the llama_multimodal handler classes and the speculative API. A stock PyPI wheel gets you a clear error naming the missing class and pointing at the fork's prebuilt wheel releases; that message is not you being unlucky, it's the honest version of "wrong wheel". This is a fiddly corner of the ecosystem, and it's where people genuinely struggle.

After that: models in the wrong directory (ComfyUI/models/LLM, refresh after adding), a projector that doesn't match the model, and memory. Vision runs are heavy - you've got the model, the projector, the KV cache for image tokens, and your diffusion model all wanting the same card. Community reports of vision GGUF runs hanging or crawling on smaller and integrated GPUs come down to batch and offload settings, which is what the hardware and prefill profile nodes exist to tune. Defaults first; tune second.

Install goes through ComfyUI Manager (search llama multimodal, install ComfyUI-llama-multimodal) or:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

Then restart. Requires ComfyUI 0.19.3 or newer. The native session path doesn't need a llama executable on your PATH - that's only for the server-backed session nodes.