llama.cpp Basic Prompt

This is the node that actually generates text. Type a prompt, wire in a server_url from a Start llama.cpp Server node, and the LLM's response comes out the response socket as a plain string - which you can feed into a text encoder, save, or pass to another node. No API key, no cloud, no Ollama sitting in the middle. It's the "any GGUF LLM as a Comfy node" thing people keep asking for, done by talking to the llama-server process the pack owns.

The name undersells it a little. "Basic" here means "no images" - the node still carries the full sampling stack, thinking-mode support, stop sequences, and workflow-chaining controls.

How it works

Basic Prompt streams tokens over the OpenAI-compatible /v1/chat/completions endpoint of your llama-server. It sends the system_prompt and prompt, streams the reply, and only reports success after a valid stream terminal marker - if the stream times out or gets cancelled, the partial text is preserved and labelled rather than silently dropped.

The thing to understand before you wire it up is keep_context. It maps to llama.cpp's cache_prompt: it reuses a matching prompt prefix in the KV cache so a repeated prefix doesn't get re-tokenized. It is not chat history, durable memory, or a session - flip it off if you expect conversation. For real multi-turn chat you'd be re-sending the full transcript each time, which is exactly what keep_context makes cheaper.

Inputs and outputs that matter

• prompt - the actual thing you're asking. Required.
• system_prompt - optional role instruction that defines behavior. For prompt-enhancement work this is where the "you are an expert prompt engineer, output only the prompt" direction goes.
• max_tokens - hard cap on output (default 2048). Reasonable default, but raise it if you're asking for long structured text.
• temperature / top_p / top_k / min_p / repeat_penalty - the usual sampling knobs. For deterministic prompt rewriting, drop temperature toward 0.5 or lower and keep repeat_penalty at 1.1.
• stop_sequences - where generation halts. One per line, a JSON string array (use this when an entry contains commas), or the legacy comma-separated form. This is how you stop the model before it pastes chat scaffolding into your prompt.
• trigger - a dependency input that orders execution. Wire something into it to make sure the LLM runs after an upstream node; enable_chaining is just a compatibility toggle now.

Outputs: response (the text), thinking (reasoning content reported separately by models that support it), and success (a Boolean you can gate later nodes on).

Wiring it

The minimal workflow is Start llama.cpp Server → this node → llama.cpp Prompt Output. Feed response into anything that accepts a string. For the classic LLM-assisted prompting loop - where the LLM rewrites your rough idea into a structured prompt and that string goes into your CLIP/text encoder - this node is the whole engine, and the reasoning about why a small obedient model beats a big thinking one applies here too.

Issues to expect

• No model loaded - the prompt node uses the running direct model; if the start node failed, success won't fire and nothing runs.
• Output has "Here is your enhanced prompt:" preamble - your model wasn't told to output only the result. Tighten system_prompt or add a stop_sequence.
• Slow first token - that's the KV cache filling on a long prompt; keep_context helps when prompts repeat across queues.