Get LLama CPP Response

The name is misleading in a useful way. "Get LLama CPP Response" does not load a GGUF, does not import llama-cpp-python, and does not touch your GPU. It is an HTTP client with a fancy face. You point it at a llama-server you already have running, it POSTs a chat request, and it hands back one string.

That's a feature, and you appreciate it after fighting the alternative. Plenty of "llama cpp" nodes link llama.cpp straight into ComfyUI's Python process, which means matching CUDA builds against wheels that don't officially exist for ComfyUI's Python version - the classic "workflow says this node is missing and I've reinstalled four times" thread. This one skips all that. If curl http://127.0.0.1:8080/v1/models works in a terminal, this node works.

Where it fits

An LLM in a ComfyUI graph is a tool, not part of the diffusion pass - it sits upstream, rewriting a rough idea into a structured prompt, captioning an image, or generating text you feed elsewhere. The local path exists for two reasons an API can't match: it's uncensored, and it's free per call. This node is a cheap way onto that path, because the model lives in a process you control entirely.

The trade is that you now run two things: llama-server holds weights in VRAM and your checkpoint holds more. A Q4 7B alongside a diffusion model is fine on 16GB, and fine on anything if you keep -ngl low and let the CPU do the work. A 30B and a video model will not share an 8GB card politely.

What it actually sends

Under the hood it builds {"messages": [...], "temperature": ..., "seed": ..., "stream": false} plus sampling fields and POSTs to <server_url>/v1/chat/completions; choices[0].message.content comes back as a plain string. The model field in the payload is literally "llama-cpp-model" - the author's comment notes the server ignores it, which is true: you get whatever GGUF the server was launched with.

Which means the prompt is the only conversation you get. No system message, no multi-turn, no history. One user turn in, one text blob out.

The inputs that matter

prompt is the one you'll actually think about, and it's multiline. Since there's no system slot, put your formatting instructions inline and be bossy: "reply with only the prompt, no preamble, no quotes."

server_url defaults to http://127.0.0.1:8080, which is llama-server's default port. If ComfyUI runs in Docker and the server on your host, 127.0.0.1 means the container - use http://host.docker.internal:8080 or your LAN IP.

temperature, top_p, top_k, min_p, repeat_penalty, presence_penalty and seed all pass straight through as sampling overrides. The defaults - top_p 0.95, min_p 0.05, top_k 20, repeat_penalty 1.0 - are the numbers the pack's own Ollama node uses for thinking models. For a small 3B, repeat_penalty of 1.0 means no penalty at all; if output loops, nudge it to 1.1 before you touch anything else.

max_tokens only gets added to the request when it's above zero, so -1 genuinely means "no cap" - the server runs until it hits the context limit or EOS.

Two things the UI won't tell you. think is collected but never makes it into the payload, so on this node it's an inert toggle (the sibling Ollama node does forward it). And the request timeout is 360 seconds, which means a slow CPU-only server raises an exception mid-run instead of hanging.

Vision, carefully

image_path and image_base64 are the optional bits, and this is where people get burned. Whichever you fill goes into an image_url field verbatim, and a bare filesystem path like /home/me/pic.png isn't a URL the server can fetch.

Feed image_base64 from the pack's Image to PNG Data URI node instead - and take its data_uri output, not base64_string. That data:image/png;base64,... prefix is the whole point; the raw string isn't a URL. You'll also need the server launched with a multimodal projector (--mmproj), or a text-only GGUF simply has no eyes.

The output

One output: response, a STRING. It wires into a text-encode node's prompt input, a string splitter, or a save-text node. It's plain chat-model text, so expect conversational crust - "Here's your enhanced prompt:" and the like. If downstream needs clean output, hold the model to a format in the prompt itself.

Install

Search "Sagado Nodes" in ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/5agado/ComfyUI-Sagado-Nodes
cd ComfyUI-Sagado-Nodes
pip install -r requirements.txt

Restart ComfyUI. The node lives under Sagado-Nodes/llm and is displayed with a stray capital, "Get LLama CPP Response" - search "llama" rather than typing the name.

The requirements file is torch, pillow, numpy, ollama. That last one is for the pack's Ollama node, not this one, and it installs the Python client - not an Ollama server. requests isn't listed because ComfyUI already ships it. Nothing here needs a compiler.

Then, separately, get a server running:

llama-server -m /models/qwen2.5-7b-instruct-q4_k_m.gguf \
  -c 8192 --host 0.0.0.0 --port 8080 -ngl 99

Add --mmproj /models/mmproj.gguf if you want the image inputs to do anything, and drop --host to the default if ComfyUI is on the same machine.

Troubleshooting

"Failed to connect to llama.cpp server at …" - the node raises this with the exact URL it tried. Check the server is up, the port matches, and 127.0.0.1 isn't pointing into a container. The console line above the error says [LocalLlamaCppVisionNode] - the class was renamed and the print wasn't, so that message is still this node.

You installed llama-cpp-python because "llama.cpp node." Uninstall it, you don't need it. This node never imports it.

Empty or truncated output - usually max_tokens against a small model's tendency to burble. Lower it and add a stop instruction to the prompt; there's no stop-string field here.