[llama.cpp] Generate (Media Sequential)

Single-shot Generate takes one bundle of media. This one takes a list and runs one independent request per item, handing back parallel result lists. Same session, same loaded model, no reload between items.

If you've ever wired a VLM into a ComfyUI graph to caption a LoRA dataset and watched it reload a 6GB model per image, this is the node you wanted. It's the batch path for the two jobs the KB says people actually use VLMs for in a graph: writing captions for training data, and seeding prompts from existing images or clips.

How it works

The media gets split into atomic items - one image each, one audio clip each, one video each (with its embedded audio attached when video_with_audio is on) - in modality order: images, then audio, then video.

Then the node runs each item as its own independent completion against the session, one at a time. Independent is the important word: the model is reused, but context is not carried across items. Item four doesn't see what item three said, and that's deliberate - it's what makes the outputs comparable instead of compounding. On a Native Session the context is reset before every completion; on a Runtime or Connect Session each item is just another stateless /v1/chat/completions call. The model unloads once, at the end, when you ask it to.

The prompt input follows the same either/or rule: one prompt is shared across every item, or a list pairs one-per-item. One value or exactly as many as there are items - anything else raises rather than guessing. So Describe this image in one paragraph. for all forty, or forty different prompts for forty different frames.

A neat side effect of that rule: connect no media at all and hand it five prompts, and you get five independent text completions from one warm model. It's a decent poor-man's batch prompt-rewriter for stripping the blank-page problem out of a prompt workflow.

The inputs that matter

session - from Create Runtime Session, Connect Session, or Create Native Session. Required.

prompt - one shared string or a list. Short and format-shaped beats chatty: "Describe this image in one paragraph for a dataset caption" gets you cleaner output than an open-ended request, for the boring reason that chat models drift into prose when you let them.

max_tokens - default 512, up to 131072. This is the budget for the whole completion. Also the budget thinking shares, if reasoning is on - which is why captioning with thinking enabled tends to truncate.

stop - a stop string. Advanced, and worth knowing about: if your model likes to start writing a new turn after the answer, a stop sequence is the cheap fix.

video_with_audio - pull each video's own audio track into its bundle.

seed - completion seed. -1 randomises.

session_unload - unload after the whole sequence rather than keeping the model warm for later nodes. The final metrics entry records that it happened.

model_profile (optional) - per-request sampling override. Note the author's own caveat: the session handler is picked at session creation and isn't changed by this.

Outputs are all lists, index-aligned: response, thinking, raw_json, metrics, media_diagnostics. There's no session output here - if you want an explicit unload, wire the session from the creating node into Unload Session separately, using its passthrough timing input to order it after the loop.

Install

Manager → search llama multimodal → install ComfyUI-llama-multimodal. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

ComfyUI 0.19.3 or later. requirements.txt installs makoto-decision (needed by the Decide nodes, harmless here). For a local model you need the llama runtime - Settings → llama-multimodal → Download llama.cpp - with the GGUF and its matching mmproj in ComfyUI/models/LLM. For vision work, the mmproj is not optional: a text-only GGUF won't see your images no matter how you wire this node.

Where people get burned

"received 3, expected 1 or 12". You connected a partial prompt list. Either trim it to one prompt or make it exactly as long as the media list.

It's serial, and that's the point. Forty images at 512 tokens each is a long queue on a consumer card, and the graph blocks for the duration. If you're captioning a real dataset, do a ten-image smoke test first and time it - you'd much rather learn the per-image cost from ten than from four hundred.

Watch metrics when output looks truncated. Per-item metrics will tell you whether you hit max_tokens or the model genuinely finished. Silent truncation reads exactly like a model that got bored.

Multi-subject captions will be wrong. This is the one weakness every VLM in this category shares - two people in frame and it mixes up who's wearing what. It's a mechanical limitation of the models, not the node, which is why the KB's advice for small datasets is still to audit captions by hand rather than trusting a bulk pass.

One model, one card. The session stays resident for the whole run. If your sampler loads next, either unload the session first or accept the swap.