[llama.cpp] Generate (Prompt Sequential)

Same request shape as plain Generate, but it takes a list of prompts and runs each one independently against the same media bundle - and because the media is pinned before the prompt, the expensive part of the work happens once.

This is the node for "same image, many questions". Score it from three angles. Ask for a natural-language description, a booru tag list, and a short prompt-shaped summary in one pass. Run four prompt-wording variants of the same caption request and pick the one your sampler likes. In a captioning pipeline it's the difference between feeding a VLM a list and reloading/refeeding the same image N times.

Why a separate node instead of a loop

The mechanism is the point. The node forces the media ahead of the prompt in the request, and sets common-prefix KV reuse on by default. The model ingests the image's vision tokens first, once; every subsequent prompt in the list shares that identical prefix, so the backend can reuse the KV cache instead of re-ingesting the image for prompt #2 through #N.

With a big image on a big model, that prefix is the slowest thing in the whole call. Reusing it is the difference between "ten captions" being a coffee break and being a chore. reuse_kv_cache is a boolean you can turn off - it's an attempt at reuse, not a guarantee, and if a run ever comes back with content that looks like it bled across prompts, that's the first knob to flip.

system is shared across every request too, which is both useful and a real constraint on what reuse can do - a vault of prompts that each need different instructions won't share a prefix.

Inputs and outputs

The inputs are Generate's: session from your session node, system, prompt, max_tokens, seed, stop, the optional images / audio / video trio with video_with_audio, plus reuse_kv_cache, session_unload, and the per-request model_profile override. prompt is where the list goes - a flat list of strings, one entry per request. It needs at least one, and every entry must be a plain string; nested lists error out.

All five outputs are parallel lists, index-aligned, so entry 3 of response belongs to prompt 3:

• response - the answers. This is what you fan out into text-encode or prompt-combining nodes.
• thinking - reasoning text, kept out of response so it never reaches your conditioning.
• raw_json - the backend reply per request, for debugging the one that looks wrong.
• metrics - per-request timings and throughput. On a reuse hit, watch them drop after the first entry; that's the cache doing its job.
• media_diagnostics - a receipt per request, feeding Media Diagnostics if you need to prove the image landed.

Wire list outputs into nodes that consume lists, or index into them if you want one specific variant as the real prompt and the rest as comparison.

Pick the right axis

The pack has two sequential generate nodes and they're easy to confuse. This one is many prompts, one media bundle - the image is fixed and the questions vary. Generate (Media Sequential) is the opposite: many images, one prompt, and it's what you want for captioning a folder of photos. Get the axis wrong and you'll spend an afternoon wondering why your second image was ignored: the prompts here all describe the same bundle, full stop.

Installing

ComfyUI Manager → search llama multimodal → install ComfyUI-llama-multimodal. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

Restart, on ComfyUI 0.19.3 or newer. Your GGUF and its matching mmproj go in ComfyUI/models/LLM/, and the in-process path needs JamePeng's multimodal llama-cpp-python fork rather than a stock wheel - if the pack can't import a handler it tells you so and links the wheels. The pack shows up as ComfyUI-llama-multimodal, ComfyUI-Ollama-ImageList, or ollama-image-list; same code, three names, renamed repo.

Where it bites

Long lists plus a big context: n_ctx on the session has to hold the image prefix and every prompt, and if you blow past it you'll get truncated or weird output rather than a clean failure. Turn session_unload on when this is the last llama.cpp step and you're about to sample, or the model sits in VRAM while your diffusion model wants it. And if the model's answers all look suspiciously similar, check whether your system prompt is forcing one style - sequential runs share it.