Llama.cpp Server Client

What it is

LlamaCppClient is not a text-generation node in the way you're probably picturing. It doesn't load a model, it doesn't run inference, and it doesn't need any GGUF weights sitting anywhere near ComfyUI. It's an HTTP client - one node that knows how to talk to every endpoint llama-server (the actual server binary from llama.cpp) exposes: completions, chat, embeddings, tokenize/detokenize, template testing, code infill, and reranking. If you've been running a local LLM through llama.cpp for anything else - a coding assistant, a chatbot, a RAG pipeline - this is the node that lets ComfyUI join that party instead of you spinning up a separate Ollama node or paying for a cloud API.

Where this fits in a real workflow: writing detailed prompts by hand for modern LLM-conditioned image models (Flux, Z-Image, anything reading natural-language instructions instead of tags) is tedious, and the community has largely moved to having an LLM write the prompt instead - it's become common enough that dedicated prompt-writing node packs now draw real attention on release. LlamaCppClient is the DIY version of that: point it at a llama-server you're already running, feed it a rough idea, and wire its text output into whatever downstream node needs a string - a CLIP/T5 text encode, a caption field, a batch of variations.

How it works

The node builds a request to whichever llama-server endpoint you pick and ships back whatever comes out. That's it - the "intelligence" lives entirely on your llama-server instance and whatever GGUF model it loaded, not in this node. Because it mirrors the server's own API almost parameter-for-parameter, it exposes essentially everything: temperature and top-k/top-p/min-p for basic sampling, DRY and XTC and Mirostat for the fancier anti-repetition and coherence tricks, JSON-schema and BNF grammar constraints for forcing structured output, chat message fields for multi-turn conversations, and separate parameter groups for embeddings, tokenization, infill, and reranking. It's genuinely comprehensive - the tradeoff for that completeness is a very long, very dense node.

Inputs and outputs that actually matter

You only need three fields to get started: server_url (default http://127.0.0.1:8080, wherever your llama-server is listening), endpoint (an 8-way choice - completion, chat_completions, embeddings, tokenize, detokenize, apply_template, infill, reranking), and prompt, the text you're sending in. Everything else in the optional list is a specific llama-server parameter - temperature, n_predict, seed, and friends for basic generation control - that you only touch once you know you need it. One trap worth knowing up front: several optional fields (stop_sequences, messages, tools, logit_bias, dry_sequence_breakers, and others) are raw JSON typed into a plain text box, not structured widgets, so a stray comma or missing bracket will break the request.

On the way out you get four outputs: response (the clean text, what you actually want most of the time), raw_response (the full JSON the server returned, useful if you need a field the node doesn't surface directly), error (empty on success, populated with a real message on failure), and status_code (the HTTP code, handy for branching a workflow on success/failure).

Installing it

Through ComfyUI Manager: search for "comfyui-llamacpp-client" and install. Manually: cd ComfyUI/custom_nodes && git clone https://github.com/fidecastro/comfyui-llamacpp-client.git, then cd comfyui-llamacpp-client && pip install -r requirements.txt, then restart ComfyUI - it'll show up under AI/LlamaCpp. That's the easy part. The real prerequisite is llama-server itself, running separately - this node ships zero model-running code. Grab llama.cpp, build or download llama-server, and start it with a GGUF model: ./llama-server -m your-model.gguf -c 4096 --host 0.0.0.0 --port 8080. If you want embeddings or reranking to work, you need to start it with --embedding and/or --reranking respectively - the endpoints those flags gate will error out otherwise.

Where people get stuck

Connection-refused errors almost always mean llama-server isn't running, or is running but not reachable at the server_url you typed - check the host/port match, especially if the server's on another machine and you left the default 127.0.0.1. Timeouts on long generations are just the timeout parameter being too low for how much you asked for (uncapped n_predict can run a while). "Endpoint not working" is almost always a missing server flag - the README calls this out directly, and it's the single most common gotcha with a server this configurable. And since one node serves eight different endpoints, it's easy to fill in fields for the wrong one - setting messages while endpoint is still completion, say. Match your fields to your endpoint choice and most of the confusion goes away.