[llama.cpp] System One Decide

Every serious ComfyUI workflow eventually needs a branch, and branches need a decision. The usual solution is to make an LLM write a sentence and then sniff the sentence for keywords - which works right up until the model writes "there is no water visible" and your keyword check flips the wrong way.

[llama.cpp] System One Decide is the pack's attempt to do this properly: one typed question, one image bundle, one probability-first answer. It's the node the builders feed.

What it does

You connect a session, a text system and context, and a question built by [llama.cpp] Build Question (choice/score) or [llama.cpp] Build Noul (true/false), plus optional images. The node POSTs to the llama.cpp server's /v1/systemone endpoint with your system and context text joined up and sent as the request's state, and any images attached as data URLs. The server returns a per-question answer with probabilities, which the node passes through untouched.

Two constraints are worth internalising before you debug anything else. System One takes images only - no audio, no video, unlike the prefill decision path. And it needs a Runtime or Connect Session; a Native Session is rejected with an explicit error, because the in-process path uses a different scoring mechanism (letter-token logits) and this endpoint doesn't exist there.

The inputs and outputs that matter

Of the inputs, session and question are the ones that define the node; system and context are two free-text fields that get labelled and concatenated - System: first, then Context:, blank lines between. They end up as one state string, so there's no separate system role here: don't write the same instruction in both and expect the model to treat them differently.

images is optional and takes a normal ComfyUI IMAGE batch. session_unload is a labelled toggle - Unload vs Keep - and the tooltip is blunt: "Unload the session after all System One requests." Set it on the last decision in a chain and the model leaves VRAM when you're done.

Four outputs come back. answer is the one that matters - a typed payload holding the type, the selected option, a value, ordered probabilities and the raw API result - and you almost never parse it by hand; [llama.cpp] Extract Answer exists for exactly this. metrics is a JSON STRING with the decision time, token usage, the model name, model_unloaded, and session bookkeeping like execution_index and model_reused - that's your first stop when something took 40 seconds and you want to know if it was the model or the request. media_diagnostics reports what was sent versus what was evaluated. And session passes the handle straight back out so you can chain another decision without re-loading the model.

One warning that's printed in the pack's own docs and deserves repeating: when media is attached, the diagnostics mark the images unverified_remote. A successful HTTP response is not proof the server looked at your picture.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

Then restart. ComfyUI Manager users can search llama multimodal and install ComfyUI-llama-multimodal instead. You need ComfyUI 0.19.3+; models and the matching mmproj projector live in ComfyUI/models/LLM; the llama binary comes from Settings → llama-multimodal → Download llama.cpp if it isn't on your PATH. The decision nodes pull makoto-decision from the pack's optional llama dependencies - Manager handles that, a bare clone might not, and the failure message names it.

Where people get burned

You pointed a Connect Session at a stock server. /v1/systemone is the specialty here, and the pack deliberately doesn't fall back to token scoring when the endpoint isn't there. A plain llama.cpp server that answers /completion fine will still 404 or 501 this node.

Native Session, again. The error is explicit, but it's the single most common wiring mistake in the family, because the Generate nodes happily accept all three session types.

Chit-chat models give you confident nonsense. A general instruct model with no vision projector will happily return a distribution over your options while never having seen the image - and, per the docs, nothing in the response tells you that happened. If the answers look uncorrelated with your pictures, check that a real mmproj is loaded, that your model actually supports vision, and that images is wired at all rather than sitting unconnected.

VRAM never comes back. This is the oldest complaint about LLM nodes in ComfyUI - people have reported models staying resident until ComfyUI restarts. Here it's on you: either flip session_unload to Unload on the final node of the chain, or wire the session output into [llama.cpp] Unload Session. Keeping the handle alive between decisions is the point; forgetting to close it is how you end up with a diffusion model fighting a language model for the same card.

Re-running doesn't cost nothing. The node is marked non-idempotent, so a repeat execution genuinely re-queries the model. Expect the queue rerun to take as long as the first pass, and budget your graph accordingly.