Start llama.cpp Server

This is the node everything else in the ComfyUI llama.cpp Suite hangs off. Nothing else in the pack works until this one has done its job: it finds your llama-server binary, launches it, loads a GGUF model into it, and only then hands you a server_url to feed into a prompt node. Think of it as a CheckpointLoader for language models, except the "checkpoint" is a whole separate process that keeps running.

The design decision that defines the whole pack: the model lives in an external llama-server process, not inside ComfyUI's Python process. That's the opposite of the in-graph GGUF loaders other packs use, and it's deliberate. Because the LLM is a separate process, you can stop it, restart it, and free its VRAM without touching ComfyUI's own model cache - which matters when you want to hand the GPU back to a diffusion model after the LLM writes your prompt.

How it works

Start llama.cpp Server resolves the executable in a specific order: the binary_path input, then the LLAMA_SERVER_BINARY environment variable, then llama-server on your PATH. It probes the binary with --version and --help, records its identity, and refuses to launch if you ask for an option the binary doesn't support. It waits for the server to be actually ready (default 60-second timeout) before reporting success.

The node is idempotent: re-queue the same config and it won't restart anything. Change the model or an effective setting and it does a coordinated restart. If a replacement preflight fails, it leaves a healthy existing server alone rather than tearing it down.

The inputs that matter

• model - the GGUF file to serve, picked from ComfyUI/models/LLM/gguf/. If the dropdown says "No models found", that folder is empty or misplaced.
• context_size - max context in tokens (default 4096). This directly sets how much VRAM the KV cache eats; too high and the server won't fit.
• gpu_layers - empty preserves the legacy "offload everything" behavior; you can also type auto, all, or a number of layers to offload.
• port - defaults to 8080. If it's taken by another listener, the pack refuses to adopt it and you'll need to change it.
• mmproj - for vision models, the matching multimodal projector. (auto) leaves discovery to llama-server, which isn't a promise it'll find your projector.

Wire server_url into a prompt node's server_url input (or just leave that empty and it finds the managed server), and success anywhere you need the workflow to wait for startup.

Installing it

Install the pack once, then install llama.cpp separately - they're independent. From ComfyUI Manager, search "comfyui-llamacpp", or:

cd ComfyUI/custom_nodes
git clone --branch master https://github.com/Setmaster/comfyui-llamacpp.git
cd comfyui-llamacpp
python -m pip install -r requirements.txt   # use your ComfyUI Python

Restart ComfyUI. Then grab a current llama-server build from the llama.cpp releases, build it from source, or use your package manager, and make sure llama-server is on PATH (or set LLAMA_SERVER_BINARY). On Windows, deploy the complete release archive into a clean directory - the CUDA builds need their companion DLLs and a separate CUDA runtime archive; swapping in only llama-server.exe breaks things. Finally, drop a .gguf file into models/LLM/gguf/.

Common issues

• "llama-server was not found" - it's not on PATH in the environment that launches ComfyUI. Verify with llama-server --version, then set LLAMA_SERVER_BINARY to the full path.
• Server starts and exits - usually a model architecture the build doesn't support, not enough VRAM/RAM, or a mismatched projector. Drop context_size, use a smaller quant, or fewer gpu_layers.
• Port already in use - the pack won't kill a random process just because it's named llama-server. Change the port or stop the other listener.

The GPU handoff tip: if you're doing LLM-writes-prompt-then-diffusion, set unload_comfy_models_before_start to evict Comfy's managed models before the LLM allocates memory. When the LLM is done, Comfy's native Unload Models action stops this pack's owned server and returns the VRAM.