Llama.cpp Python Server

An LLM needs a server before anything can generate, and this node is what stands one up for you. It spawns llama-cpp-python's built-in HTTP server - the python -m llama_cpp.server command - as a subprocess, waits until it answers, and hands you the server_url string that the pack's Llama.cpp Generate node plugs into. Two nodes, one job: this one is the plumber, the other one is the talker.

The mechanism, straight from the code: it builds a command line with --model, --host, --port, and --n_gpu_layers and launches it with ComfyUI's own Python (sys.executable). Then it polls the /v1/models endpoint until the server responds, and only then returns the URL. The server is a module-level singleton: once it's up, rerunning the node reconnects to the same process instead of spawning a second one, and it's terminated when ComfyUI exits. That's good behavior - you don't want a dozen llama.cpp servers piling up.

The inputs that matter

• model_path - defaults to models/llama/model.gguf. That's relative to ComfyUI's working directory, so it means ComfyUI/models/llama/model.gguf. Drop a GGUF there. This is the "bring your own model" part: unlike some flashy LLM nodes, this pack never auto-downloads anything, and a GGUF quantized model is what makes a chat LLM fit on the same card as your diffusion model in the first place.
• n_gpu_layers - default -1, which means all layers on the GPU. If you're sharing VRAM with the diffusion model, dial this down to something like 20–30.
• auto_start - on spawns the server for you; off just returns the URL so you can run the server yourself (useful if you want to manage it outside ComfyUI).
• host / port - 127.0.0.1:8000 by default.

Output: server_url, wired straight into Llama.cpp Generate.

The gotcha that hangs the UI

Here's the trap, and it's visible in the code: the startup poll loop has no timeout. If the subprocess fails immediately - which is exactly what happens when llama-cpp-python isn't installed, or the model path is wrong - the node loops forever on "Waiting for LLM to load..." and the queue just sits there. The time_out input exists in the schema but isn't what saves you; the blocking wait is a separate loop that never gives up. So the rules of thumb:

1. Install llama-cpp-python first, into the same Python environment ComfyUI uses. There are no official wheels for ComfyUI's Python, so grab a prebuilt wheel (JamePeng's llama-cpp-python releases are the community standard; pick the one matching your Python and CUDA) and pip install it into the env, or compile with CMAKE_ARGS.
2. Watch the console, not the canvas. If you see the "Running Server" / CMD printout followed by nothing and no model output, the server died instantly - check the module install and the model path.
3. Big GGUFs load slowly. The first load of a 7–8B model can take a while, and the node looks frozen while it does. If the console shows the process running, give it time.

Installing the pack

ComfyUI Manager: Custom Nodes Manager → search ComfyUI-LLMNodes → install. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/LatentGEN/ComfyUI-LLMNodes

Restart afterward. The pack's requirements.txt is empty and it declares no dependencies - the llama-cpp-python wheel is the real install, and it's on you.

Why bother running local at all?

Because it's free per call, offline, and uncensored - the reasons the community runs small GGUFs for prompt work instead of paying an API, per the KB's analysis of this exact pattern. And this server-node approach is the classic two-process setup: llama.cpp serving on localhost, the graph talking to it. It works, it's just the fiddlier of the two local-LLM paths (the Ollama route is the easier on-ramp, the in-graph GGUF loader the direction the field is drifting).

One last honest note: this pack is brand new (late 2025), the README is empty, and the author has no community footprint - so treat the node as an early release. The core mechanism is sound, but the missing timeout is the kind of edge you're signing up for.