LLM (llama.cpp server)

The name is a bit of a lie, in the good way: "LLM (llama.cpp server)" sounds like something that talks to a service you have to set up and pay for. It doesn't call any API and needs no key. Wire in a model handle, type a prompt, and a local llama.cpp server that this pack started itself does the work on your own GPU. This is the node that makes "run any GGUF chat model directly in ComfyUI" real, without an Ollama install in the middle.

Why you'd reach for it: the LLM-in-the-graph pattern. You type a rough idea and a small local model rewrites it into a structured, model-appropriate prompt that feeds your text encoder - which makes sense when the encoder is itself an LLM reading an instruction. Or you caption an image for a LoRA set, or turn a video-model prompting guide into next-scene prompts. The payoff of local over an API is the same one that makes this whole corner of the ecosystem tick: uncensored, offline, free per call. The classic mistake is reaching for a big reasoning model to do this. Rewriting a rough idea wants small-and-obedient; a reasoner just spends tokens and leaks its scratch-work into your prompt. An 8B, ideally an abliterated one, is the right tool.

How it works. The Load node hands this a LLAMACPP_MODEL handle, and generation is a plain HTTP POST to http://127.0.0.1:<port>/v1/chat/completions - the OpenAI-compatible endpoint llama-server exposes. ComfyUI itself never loads the model; that's the whole design of this pack. The inputs that matter:

• llamacpp_model and prompt are required - the model handle from "Load llama.cpp Model", and your prompt.
• system_prompt sets the model's framing. image, audio, video are the multimodal side: an image tensor gets base64'd to a PNG data URL, audio becomes a WAV, video gets sent as mp4 - but all three need a model with the matching mmproj projector or they're just ignored bytes.
• skill accepts a "Load llama.cpp Skill" handle and injects it as system prompt.
• history takes a JSON array of chat messages, which is how you keep a conversation going across calls - paste the previous turn back in.
• The sampling knobs are the usual suspects: temperature (0.8), top_p (0.95), top_k (40), max_tokens (1024), repeat_penalty (1.0), seed (-1 for random).

Outputs: text and reasoning. The second one reads the reasoning_content field that DeepSeek-style and Qwen reasoning models emit - if yours doesn't, it just comes back empty. Nice for separating the model's thinking from its answer.

The lifecycle flags matter more than they look. auto_start (default on) ensures the server is up before the call. release_after_use (also default on) kills the server the moment generation finishes so VRAM is freed for the diffusion pass. That's great for one-shot prompt enhancement. It's a trap if you're captioning a batch of images in a loop - every call pays the server cold-start, and you don't get your VRAM back until the queue ends anyway. Flip release_after_use off for loops, keep it on when the LLM is a one-off upstream of a sampler.

Troubleshooting. history JSON error means your JSON didn't parse as a list of {role, content} objects. LLM HTTP error 503/404 means the server isn't up on that port - check the port matches the Load node, and that the server actually started (the log lands in llama_server.log next to nodes.py in the pack folder). If the model ignores your image, you're missing the mmproj - the node can't conjure vision a model doesn't have.

Install is the same pack-wide story: grab llama-server from the llama.cpp releases, drop a GGUF model in a folder, edit config.json roots, done - no pip install llama_cpp_python, no Python deps at all. One honest caveat for a brand-new pack with zero installs yet: it's a small, readable codebase and it doesn't auto-download anything, so spend the one minute reading it before you trust it with a workflow - that's the standing rule for LLM nodes in this ecosystem.