LocalLLM Prompt Generator (GGUF)

The name is the honest part: text in, text out, using a GGUF language model on your own GPU. No key, no endpoint, no per-call bill. You type "a cozy cabin in a snowy forest" into user_prompt, this node hands that to a 4B or 24B abliterated model, and the text output goes into a CLIP Text Encode like any other string.

Why you'd want this instead of just writing the prompt

Because in 2026 you're often already writing prose. Krea 2's encoder is Qwen3-VL, Z-Image and Klein encode with the plain Qwen3 LLM - these bases were built to read instructions, and Krea's own guidance is to describe the image the way you'd describe it to a person, long and specific, with quoted words for anything you want rendered as text. That's a job a language model does faster than you do at 1am. On a tag-trained base (Illustrious, Pony), just tell it to emit comma-separated booru tags instead.

The local part matters more than the "AI" part. A frontier API writes better English, but it filters, costs per call, and needs the internet. An abliterated local model is uncensored, offline and free at the thousandth call, which is exactly why people run one for prompt rewriting. What you're buying is the removal of the blank page, not a better writer.

How it actually works

The model dropdown is a catalog, not a folder scan. models.json ships two families - Mistral-Small-3.2-24B abliterated (vision-capable, 14 quants, Q4_K_M at 14.4 GB marked recommended) and Qwen3-4B abliterated (text-only, 21 quants, Q4_K_M at 2.5 GB) - with the download size in the label, because that's the number that decides whether it fits. First run fetches the GGUF from HuggingFace with a resume-capable progress bar. A .gguf you drop into ComfyUI/models/LLM/ yourself shows up as Local: <filename>, and custom_models.json (copy the .example) adds your own repos to the dropdown.

Then it's a two-message chat - your system_prompt, then user_prompt - through llama.cpp's chat completion. The cleanup matters as much as the generation: <think> blocks are stripped, and if the model wrapped its answer in <prompt>...</prompt> (the convention the bundled prompt library uses) you get the last block only, so preamble and markdown scaffolding don't land in your conditioning. Forks that expose a reasoning budget get reasoning_budget=0.

The inputs worth touching

• model - start with Qwen3-4B Q4_K_M if you share the card with SDXL. The 24B is nicer at long video prompts, and about 12 GB heavier at the same quant.
• user_prompt - your idea. text_input is socket-only and gets appended to it with a blank line, which is how you chain a caption into a rewrite.
• system_prompt - the author ships a solid "expert prompt engineer for SD/SDXL/Flux, output only the prompt" default. It's ignored when you connect system_prompt_input.
• temperature - ships at 1.0, and the bundled prompt library assumes 0.5–0.6. Drop it for anything structured.
• max_tokens 6144 and n_ctx 8192 - the tooltip is right that reasoning models burn tokens before answering and that H3 video prompts need 4096+.
• suppress_thinking - appends /no_think to the user message (a Qwen3 thing; harmless elsewhere).
• n_gpu_layers -1 offloads everything; unload_after_run frees VRAM the moment it's done.

Output is a single text → CLIP Text Encode, or Show Text while you tune the system prompt.

Install

Manager → Custom Nodes Manager → search Slarti LLM Nodes, or:

comfy node install slarti-llm-nodes
# or
cd ComfyUI/custom_nodes && git clone https://github.com/Slartibart23/ComfyUI-Slarti-LLM-Nodes.git

requirements.txt is deliberately almost empty: llama-cpp-python is not in it, because a plain pip install llama-cpp-python gives you a CPU-only build, which is unusable for a 24B. On first run the pack checks for GPU offload and installs a CUDA build into the same Python ComfyUI runs on - cached wheel in ComfyUI/models/LLM/.wheels, else a prebuilt wheel from the JamePeng fork matched to your OS/Python/CUDA, else a source build. Restart once afterwards. The startup log tells you what you got:

[LocalLLM] ComfyUI-Slarti-LLM-Nodes v2.0.1 loaded (4 nodes) - llama-cpp-python 0.4.0: GPU

Where people get burned

The startup line says CPU-only. Check the lines above it for ggml_cuda_init: found 1 CUDA devices. Since v2.0.1 the pack copies missing cudart/cublas DLLs out of torch/lib, but that needs your PyTorch CUDA major version and the wheel's to match (both cu13x, say).

Windows [WinError 5] Access denied during auto-install. llama-cpp-python was already loaded and Windows locks its files. Close ComfyUI and install the cached wheel by hand: python_embeded\python.exe -m pip install --force-reinstall --no-deps "ComfyUI\models\LLM\.wheels\<wheel>.whl".

missing tensor 'blk.NN...' or unknown model architecture. Your llama.cpp is older than the model. Grab a JamePeng fork wheel - the Assets section is collapsed by default - matching Python (cp312) and the CUDA version from nvidia-smi.

The first run takes a minute and looks hung. That's a 14 GB download plus model load. After that the instance is cached, keyed on model + mmproj + n_ctx + n_gpu_layers - change n_ctx and you pay the reload again. A 24B plus SDXL won't coexist in 24 GB; use unload_after_run or the Unload node.

One honest note. A pack that downloads multi-GB weights and installs Python wheels by itself is the exact shape of thing that turned into the ComfyUI_LLMVISION malware incident. This one is MIT and does what it says - but "one click, gets its own model" is a reason to glance at the repo, in every pack, forever.