[llama.cpp] Connect Session

This node loads nothing. It downloads nothing. It takes a URL, points your graph at a llama.cpp HTTP server that something else is already serving - your own llama-server in a terminal, a beefy box on the LAN, whatever - and hands you a session handle the rest of the pack's generation nodes can talk through.

That's the whole pitch, and it's a good one. The alternative is [llama.cpp] Create Runtime Session, which spawns its own server per run. Connect is for when the model is already resident and you don't want ComfyUI starting and killing a process every queue. If you're running a model server for other reasons too - an editing app, a writing UI, a dataset tool - Connect is how ComfyUI joins that party instead of insisting on its own.

How it works

The moment the node executes it does two HTTP calls against the server: a health probe at /health that must come back {"status":"ok"}, and a model listing at /v1/models, which is what fills the available_models dropdown. Anything after that - POST /v1/chat/completions - happens in the Generate and Decide nodes, not here. So this node's job is: fail fast on a bad URL, then carry a session object.

That's also why it's marked not-idempotent inside the pack: it re-runs and re-probes on every queue instead of caching. A dead server becomes a run-time error, which is honest but means you find out at execute, not at graph load.

The inputs that matter

url - the base URL, HTTP or HTTPS only. The default is http://127.0.0.1:9931, which is a convention, not a law: vanilla llama-server defaults to port 8080 and LM Studio's server to 1234. Put whatever your server actually printed on startup.

available_models - the combo the node populates from the server. Pick your model here; it gets copied into model.

model - a string, and it's the value actually sent with every request. It has to match a model ID the server reports, or the server will reject the request even though everything else looks right. Selecting from available_models is safest; typing by hand is how you end up off by a path segment.

api_key - optional, for a server started with --api-key. Leave empty for a plain local one.

One output: session, typed OLLAMA_IMAGE_LIST_LLAMA_CPP_SESSION (blame the pack's earlier life as an Ollama node list for the type name). Wire it into Generate, Generate (Media Sequential), Decide, Decide (Media Sequential), or Unload Session.

Install

ComfyUI Manager → search llama multimodal → install ComfyUI-llama-multimodal, then restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-llama-multimodal.git

Needs ComfyUI 0.19.3 or newer. The pack's requirements.txt pulls in makoto-decision, the library the Decide nodes score with; the heavy optional piece is the multimodal llama-cpp-python fork, and you only need that for the Native session path, not for Connect. For this node, your "dependency" is really the server: start it yourself with an mmproj loaded if you want images to work, e.g.

llama-server -m models/LLM/your-model.gguf --mmproj models/LLM/your-model-mmproj.gguf -c 8192

Where people get burned

It has to be a llama.cpp server, not "OpenAI-compatible" in the loose sense. The pack's server-session code needs /health, /v1/models and /v1/chat/completions, and if you later attach a Decide node it also wants /apply-template, /tokenize and grammar-constrained /completion. Ollama at port 11434 will not work here - that's a different API, and it's why the same pack ships separate Ollama nodes.

A model without a projector can't see your images. Connect can't add vision; if the server was started with no --mmproj, you get a text model and your images input quietly does nothing useful. That mmproj/multimodal pairing trips people up in every tool that has it - the projector file is not a CLIP, it doesn't go in the clip folder, and renaming it to something "tidy" breaks discovery (r/comfyui spent eighteen comments on exactly that).

A wrong model string and a wrong port look identical from the node's side - both are "server said no". Check /health in a browser first; if that answers ok, the blame is almost always the model ID.

One last thing that applies to every node in this pack: a custom node is arbitrary Python running as you the moment ComfyUI loads it. An "LLM vision" node called ComfyUI_LLMVISION once shipped infostealer code through normal channels and ended in a federal prosecution, and nothing about node installation changed afterward. Install packs you can read.