[llama.cpp] Prefill Decide (Prompt Sequential)

Most LLM-in-a-graph nodes give you prose. This one gives you a choice, and then tells you how confident it was. If you've ever wanted "which of these five style tags fits this image" answered as data instead of as a paragraph you have to regex, this is the node.

It's the batched form of Decide: you supply paired lists of contexts and questions, and every pair runs independently against the same complete media bundle. Ten routing decisions over one image, one model load, one image ingest.

How the decision actually happens

The question comes in as a typed payload from Create Question From Input - a question string plus a menu of 2 to 26 unique answers, one per letter. The backend then forces the model to answer with exactly one label and nothing else: no explanation, no preamble. On a native in-process session that's done as a prefill scoring pass over the label tokens; on a server-backed session it's a constrained completion.

Two consequences worth knowing before you build on it. First, you're reading token probabilities over your own answer labels, not free-form text, which is why the second output exists. Second, native prefill scoring is deterministic - the author's own tooltip on seed says as much, so a seed does nothing for a native session and only applies to runtime completion.

Inputs and outputs

Required: session, system, context, question (the typed payload), video_with_audio, seed, reuse_kv_cache (on by default), session_unload.

context and question are where the "sequential" part lives. Give each one a list of ten and you get ten decisions paired in order. Give one of them a single value and it broadcasts across the other list. What it won't do is guess: a length that's neither 1 nor N is an error, so a mismatch in your list plumbing is loud.

Optional: images, audio, video - the shared bundle every decision sees - and model_profile as a per-request override. Media only goes into the context; the question and answers stay text.

Outputs, all lists except the last:

• selected - the winning label per decision. This is the one you switch on.
• probabilities_json - the full distribution over your labels, in your answer order, per decision. This is the underrated output. A 51/49 split and a 99/1 split produce the same selected value and mean completely different things; print this before you trust the routing.
• metrics - per-decision JSON with timings, the operation, the seed, and unload state.
• media_diagnostics - a receipt per decision, for Media Diagnostics.
• session - passed forward single (not a list), so you can chain another decide or generate step on the same loaded model.

What it's good at, and what it isn't

Good: routing and scoring over text decisions - which prompt template, which negative preset, which of six caption candidates to keep. Cheap, repeatable, and the KV reuse means a shared context prefix isn't recomputed per decision; reuse_kv_cache is doing real work here because your media bundle and context often match across pairs.

Not good: asking a small local model to be a judge of image quality. It picks a letter, and it does that in a few hundred tokens of vision context. Where the content is subtle, look at probabilities_json before you trust selected - the honest answer is often "this model has no idea", and a distribution near uniform is exactly how it says so. That's a much better signal than a coin flip dressed as a decision.

Installing

Manager: search llama multimodal, install ComfyUI-llama-multimodal. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

Restart; needs ComfyUI 0.19.3+. A vision decision needs a GGUF and its matching mmproj in ComfyUI/models/LLM/, loaded by Create Native Session. The in-process path wants JamePeng's multimodal llama-cpp-python fork - plain wheels throw a handler-class error that names the wheel source. And yes, the pack answers to three names: ComfyUI-llama-multimodal (Manager), ComfyUI-Ollama-ImageList (URL), ollama-image-list (CLI).

Traps

Feeding it something other than a Create Question From Input payload gets you an explicit complaint about where question must come from - it re-validates, so a JSON string node won't sneak past. Unloading too eagerly kills the reuse benefit; leaving session_unload off until the last decision in the chain is the right default. And if you're deciding about media that never arrived, you'll get confident labels about nothing - check media_diagnostics on the first run, same as every other node in this family.