Llama.cpp Generate

If you want a local LLM to turn your rough idea into a proper image prompt, this is the node that actually talks to the model. It takes a server_url (from the pack's Llama.cpp Python Server node), a prompt, and some sampling knobs, then posts to the OpenAI-compatible /v1/chat/completions endpoint on a llama.cpp server and returns the model's text.

First, the thing most people miss: this is not a general chat node. The system prompt is hardcoded in the source - "You are creating a propmt for a text-2-image model. Only respond with the prompt, do not add any extra commentary." (typo in the original, not mine). So its job is prompt enhancement: you type "cyberpunk samurai in the rain, neon" and it hands back a richer, structured prompt you can feed to a CLIP encoder. That is, per the community consensus, the single most common reason anyone puts an LLM in a ComfyUI graph at all - and running it locally means it's free per call, offline, and doesn't filter. A small 8B GGUF is genuinely enough for this job; the KB's own analysis is that this kind of rewriting wants a small, obedient model, not a reasoning model that leaks its chain-of-thought into your prompt.

The inputs that matter

• server_url - where the llama.cpp server lives. Usually the output of Llama.cpp Python Server in the same workflow.
• prompt - your rough idea, multiline.
• max_tokens - default 256, plenty for a prompt.
• temperature - default 0.7; lower it if the model gets creative in ways you don't want.
• seed - default 1000.

Optional: top_p (0.95) and stop. Output is text, which you wire into a text encoder or a prompt builder.

How it behaves - the gotchas

• It caches on (prompt, seed). Same prompt plus same seed returns the previous answer without hitting the model. To force fresh output, change the seed or edit the prompt. This trips people constantly: they tweak a workflow, run, and get the same text back.
• The output isn't always clean text. The response formatter strips special <|channel|>analysis<|message|>... tags and **bold** markers, and what actually comes out of text is a dict - {"thinking": ..., "message": ...}. Wire it into a plain text viewer and you may see a Python dict repr instead of prose. That's a quirk of the current code, not a wiring mistake on your end.
• It swallows errors into the output. If the server is down or the model path is wrong, the node doesn't raise - it returns a string like [llama.cpp error] ConnectionError .... So keep an eye on the ComfyUI console; "text" that starts with [llama.cpp error] is the node telling you the plumbing broke.

The dependency you actually need

This node does nothing without llama-cpp-python installed into the same Python environment as ComfyUI, plus a GGUF file. The pack's requirements.txt is empty and Manager won't install it for you - this is the step that catches everyone. The problem: llama-cpp-python has no official wheels for the Python ComfyUI ships, so the community standard is to grab a prebuilt wheel (JamePeng's llama-cpp-python releases are the usual source - pick the wheel matching your Python version and CUDA) and pip install it into ComfyUI's env directly, or compile it yourself with CMAKE_ARGS. On Windows portable installs, that's .\python_embed\python.exe -m pip install llama_cpp_python-...whl. It's the fiddliest install in this whole pack, and it's the same pain anyone with a llama.cpp node hits regardless of which node pack you picked.

Installing the pack

ComfyUI Manager: Custom Nodes Manager → search ComfyUI-LLMNodes → install. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/LatentGEN/ComfyUI-LLMNodes

Then restart. Again: the pack itself installs clean, the llama-cpp-python dependency is the part that won't.

Troubleshooting

• Output starts with [llama.cpp error] - server not running, or llama-cpp-python not installed. Check the server node and the console.
• Same answer on every run - cache. Change seed or prompt.
• Dict-looking output - expected with the current formatter; if it bothers you, pipe it through a node that can pull out the message value.

Worth noting the whole pack is young - brand-new repo, empty README, no author footprint in the community. The cache and the dict output are the sort of rough edges you get with a first release, but the core path - local GGUF in, decent image prompt out - does work once you get past the wheel dance.