[llama.cpp] Prefill Decide (Media Sequential)

Plain [llama.cpp] Decide answers one question about one bundle of media. This variant does it in a loop: it walks your media list one item at a time and returns a decision per item, as parallel lists you can index against each other.

The honest use case is triage. You load sixteen frames from a video, ask "is this frame a hard cut?" and get sixteen yes/nos back. You load a folder of reference renders and ask "is there a human face here?" to filter a batch before captioning. You're not asking the model to be right about nuance - you're asking it to make a cheap, consistent call at scale so the next node can be smarter.

How it works

The node splits media into atomic bundles and runs one full decision per bundle:

• each image is its own bundle,
• each audio clip is its own bundle,
• each video is its own bundle (with its embedded audio attached when video_with_audio is on).

Order is modality order: images, then audio, then video. That matters when you're zipping outputs back to filenames.

context and question can go either way: pass one and it's broadcast to every item, or pass a list and each item gets its own. Length mismatch is an error rather than a silent pad - the node refuses to guess which caption belongs to which frame. The scoring itself is the same machinery as the single Decide node: single-token candidate labels, either prefill logits on a Native Session or a grammar-constrained one-token completion on a Runtime/Connect Session, with probabilities renormalised across the candidate set.

It reuses one loaded session the whole way through, so you pay the model load once, and it serialises the requests. That's the right trade for a local card - you're not getting throughput, you're getting "the model isn't reloaded sixteen times".

The inputs that matter

session - required, from Create Runtime Session, Connect Session, or Create Native Session.

question - same payload type as Decide, so it comes from [llama.cpp] Create Question From Input. For per-item questions you connect a list of those payloads.

context - one string for everything, or one per item. If your question doesn't depend on per-item text, leave it empty and keep the prompt in question.

images / audio / video - the lists being iterated. Connect the same image batch you'd otherwise feed one at a time.

video_with_audio - attach each video's own audio track to its bundle.

seed - runtime completion seed; Native prefill scoring is deterministic and ignores it.

session_unload - close the session after the whole list is done, rather than after each item.

Outputs are list-typed and index-aligned: selected (the original answer strings), probabilities_json, metrics, media_diagnostics, plus session as a single unwrapped handle for chaining.

Install

Manager → search llama multimodal → ComfyUI-llama-multimodal, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

ComfyUI 0.19.3 or later. requirements.txt brings in makoto-decision, which the Native Session scoring path needs. For a Runtime Session, get the llama runtime via Settings → llama-multimodal → Download llama.cpp, and put the GGUF plus its matching mmproj in ComfyUI/models/LLM. Video comes out of the VIDEO input directly - no separate ffmpeg install required by these nodes.

Where people get burned

Count your items before you queue. Sixteen images and an empty context is fine; sixteen images and a three-item context list is a hard error, by design. If you're building the context list with another node, make absolutely sure it's the same length as the media list - that's where this pattern breaks in practice.

List outputs are only useful if something downstream consumes lists. If your next node takes a single STRING, ComfyUI will either iterate it for you or hand you a list you didn't expect. Wire the list into a text-collector or the pack's diagnostics before you assume the shape.

Per-item decisions are not comparable confidence scores. The number in probabilities_json is a renormalised distribution over your candidates - great for "which of these two labels", useless as "how sure is the model across items". If you need that, look at whether the probabilities are near-tied; a 0.51/0.49 split is the model shrugging, and a triage filter built on that will be noisy.

Same one-token rule as Decide. Candidates must each encode as a single token, otherwise the run fails rather than guessing. Keep answers short and distinct, and keep the count between two and twenty-six.