Llama.cpp Server Node

The name is a little bit of a lie, and that's the thing to understand first. Llama.cpp Server Node doesn't run a model. It's a client - a front-desk clerk that dials up a llama.cpp server you're already running and hands the answer back into your graph as plain text. It finds which model the server has loaded and talks to it over the same OpenAI-compatible /v1/chat/completions endpoint every local-LLM tool uses. No API key. Nothing downloads. If the server isn't running, the node is a dead phone.

Why reach for it? An LLM in the graph is useful for the jobs the KB's LLM-in-ComfyUI essay maps: rewriting a rough idea into a structured prompt, captioning a LoRA dataset, or writing dialogue for a video clip. What makes this one different is that it doesn't try to be the whole stack. The popular prompt-enhancer packs ship their own GGUF models, juggle VRAM, and block network calls at import - but they're also the category that shipped malware once. "Bring your own server, let this node just talk to it" is a leaner, more inspectable posture. You already trust llama.cpp; this is a thin socket between it and the graph.

How it works

Each run POSTs a chat-completions request to your server with a messages array - system prompt first, then your request - plus the sampling knobs in the payload. Two details make it worth more than a hand-rolled API node:

• Vision. The optional images input accepts a regular ComfyUI IMAGE tensor. Each image is converted to JPEG, base64-encoded, and sent as an image_url content block. That only works if the served model is actually multimodal - a Qwen3-VL-class GGUF, not a text-only Llama.
• Reasoning extraction. If the server returns reasoning_content (llama.cpp does for reasoning models), or the model wraps its chain-of-thought in <think>...</think>, the node strips it out and routes it to a separate thinking output. That's rarer than it should be - most nodes just let CoT bleed into your prompt, which is a known trap. Here the answer and the deliberation arrive cleanly split.

Session history is kept per node until you flip reset_session, so you get multi-run conversations for free.

The inputs that actually matter

The list is short enough to skim, and you'll touch maybe four fields in practice:

• url - where the server lives. Default http://127.0.0.1:8080 is the llama.cpp default port; change it if your server is elsewhere.
• model - a dropdown populated by the Reconnect button the pack adds to the node. It queries the server's /v1/models, /models, or /slots endpoints and lists what's actually loaded. Auto (Detect Active Model) grabs the first one, which is fine when you serve a single model.
• system_prompt and user_request - the actual conversation; both multiline.
• keep_alive + keep_alive_unit - how long the model stays resident before the node schedules an unload via /model/unload. Default is 0, which means the model unloads right after every request. If you're chaining multiple LLM steps, bump this to a few minutes or you'll pay cold-start reload every single call.

Sampling controls have sensible OpenAI-style defaults and seed pins reproducibility. Outputs are two strings: result (the answer) and thinking (the reasoning, if any) - wire either into a text preview node, a prompt constructor, or downstream logic.

Installing it

ComfyUI Manager is the easy path: search for "ComfyUI-llamacpp-server", install, restart. Manual install is the usual two-liner:

cd ComfyUI/custom_nodes
git clone https://github.com/Localsmile/ComfyUI-llamacpp-server

Then restart. The requirements.txt is nearly a no-op - a single diskcache entry the node never imports, since requests, numpy, PIL, and aiohttp are all already part of ComfyUI. There's also an install.py that sets up llama-cpp-python from the community cuBLAS wheel index - handy only if you don't already have a server and want the llama_cpp.server route to running one. It's a helper, not a requirement.

Where people get burned

• The node won't connect. The dropdown shows Error: or Connecting... and won't list models. In 99% of cases the server isn't running or the URL is wrong. Start your server, confirm it's reachable in a browser, then hit Reconnect - the list only populates when the node can reach /v1/models.
• It's slow in a chain. Default keep_alive = 0 means reload-on-every-call. Set it to a few minutes for multi-step pipelines.
• Images go nowhere. The images input silently does nothing useful if your served model isn't vision-capable. Check the server log for what's actually loaded.
• Long generations time out. The request has a hard-coded 120-second timeout, so don't push max_tokens toward the 32768 cap on a slow machine.

Small pack, single purpose, and it does it correctly: local, uncensored, offline LLM text and vision from a server you already run, with chain-of-thought kept out of your prompts. If you're already living in llama.cpp-land, this is the node you'd reach for.