[llama.cpp] Generate

This is the plain one, and the one that does 90% of the work: one session, one system message, one prompt, optional media, text back. Caption an image. Write a Wan or LTX prompt from a reference frame. Translate a prompt. Rewrite a rough idea into the structure your checkpoint's encoder actually wants.

To be clear about where this sits: it's the LLM as a tool bolted into the graph, not the text encoder inside your checkpoint. Those are different things and the KB's llm-in-comfyui.md hammers the distinction for good reason - one of them you install and wire, the other is frozen inside the model file and runs every generation no matter what.

What it does mechanically

It takes session from Create Native Session (or a server-backed session), hands over your system and prompt plus whatever media you connected, and runs a single request. What you get back is split into pieces rather than dumped as one string:

• response - the actual answer. This is what you wire into your CLIP text encode or prompt-combining node.
• thinking - reasoning-channel text, separated out. This matters more than it sounds. The classic local-LLM-in-a-graph failure is a chat model leaking "Here is your enhanced prompt:" or a thinking block straight into your conditioning, where it becomes literal prompt tokens. The pack routes that text to its own output instead, so your sampler never sees it.
• raw_json - the uncut backend reply, for when the response looks truncated or weird and you want to see what the model actually said.
• metrics - JSON with timings and throughput, plus whether the model was unloaded after the response.
• media_diagnostics - the media ingestion receipt. Wire it into Media Diagnostics when you need to prove the model saw your image at all.

Inputs you set

prompt and system are the two you'll touch every time. system is where instructions belong - "you write camera-direction prose for video models, no preamble, no markdown". A well-set system message does more for output cleanliness than any sampler tweak.

max_tokens defaults to 512, max 131072. 512 truncates a detailed video prompt mid-sentence more often than you'd think. If output looks cut off, raise it before you blame the model.

seed at -1 means random per run. Set it when you want to compare prompt variants fairly instead of comparing two coin flips.

stop is an advanced string for cutting the model off at a marker when it insists on continuing past the answer.

images, audio, and video are optional media inputs, with video_with_audio a boolean for whether a video's embedded audio track gets passed along too. What actually works depends on the model - image support needs the mmproj, audio and video are narrower still.

session_unload closes the session after this request. model_profile overrides sampling and reasoning for this one request; note the handler is fixed when the session was created, so it's a knob for sampling, not for swapping model behaviour mid-graph.

A sane first graph

Create Native Session → Generate → your negative/positive text encode, with a preview text node hanging off response for the first few runs. Read the actual output before wiring it into a sampler - you're looking for the model's chat habits showing up in the text, not for whether the picture is nice.

Installing

Manager: search llama multimodal and install ComfyUI-llama-multimodal. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

Restart, and make sure you're on ComfyUI 0.19.3+ or these nodes never register. Put your GGUF (and its matching mmproj) in:

ComfyUI/models/LLM/

Two dependencies are on you, not the installer. A server session needs a llama executable - Settings → llama-multimodal → Download llama.cpp installs one into the pack if you don't want to manage it on PATH. And the in-process path needs JamePeng's multimodal llama-cpp-python fork; a stock wheel fails with a message naming the missing handler class and where to get wheels. One naming quirk: this pack also answers to ComfyUI-Ollama-ImageList and ollama-image-list. Same code, renamed repo.

Common issues

"session must be a Llama.cpp Create or Connect Session output." You wired something else into session, or the session node errored and you're seeing the downstream complaint.

The answer ignores your image entirely. Not an error, just a confidently wrong caption about nothing. Check media_diagnostics - usually a missing or mismatched mmproj, or a model that isn't actually vision-capable.

The response reads like a chatbot. Your system prompt is too loose. This pack separates thinking out, which handles the worst offender, but preamble is still on you to instruct away.

It's slow. You're running a language model and a diffusion model on one card. Unload between stages, use a smaller quant, or drop to a 3-4B model - for short structured rewriting that's usually enough, and the KB's verdict on this job is small-and-obedient over big-and-clever.