GGUF_LLM

This is the local, no-API-key sibling of OpenRouter LLM in this pack, and for prompt optimization work it's the one you want if your GPU can breathe. GGUFInference (displayed as "GGUF_LLM") runs quantized LLMs - Qwen3-4B, Qwen2.5-VL-7B, and similar - through llama-cpp-python, right inside ComfyUI. The "GGUF" part is the good news: because the models are quantized, a 4B model runs fine on an 8GB card, and the vision variants will look at your images too. It's the same trick the ComfyUI ecosystem uses to fit Flux-class models on smaller cards, applied to LLMs.

How it works

The node scans your text_encoders and clip model folders for .gguf files and lists them in the model dropdown. Pick one (or use the built-in "Download: …" entries, which pull a suggested model from HuggingFace with one click) and the node loads it via llama-cpp-python, offloading layers to GPU when CUDA is available. A few things the code does for you:

• Vision mode - models with "vl" in the filename are treated as vision-language models. Connect image_1/2/3 plus a mmproj_file and it'll describe what it sees; without those it falls back to text-only.
• Auto-repair - the auto_install_llama_cpp checkbox (on by default) reinstalls or fixes a broken llama-cpp-python install. It prioritizes a "basic" build for compatibility, which the tooltip is candid about.
• Cleanup - <think>...</think> reasoning tags get stripped from output, which keeps prompt-optimizer output usable downstream.

The inputs that matter

• model - GGUF file or a download option. Default is a QwenVL download.
• prompt - your request.
• prompt_template - the pack's templates (image_to_prompt.md is the default, which suits the vision models).
• max_tokens, temperature, top_p, top_k - standard sampling controls; defaults (3072 / 0.7 / 0.9 / 40) are sane.
• keep_model_loaded - leave it off for occasional use (model unloads after each run, freeing VRAM for the image model), turn it on if you're looping LLM calls and can spare the memory.
• seed - 0 for reproducibility.

Outputs: text (the response), used_seed (the seed actually used - handy when you want to reproduce a run), and prompts (the list of rendered template prompts, for debugging).

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/dseditor/ComfyUI-ListHelper

Restart ComfyUI; the node lives under ListHelper/LLM. The pack's own requirements (regex, accelerate, reportlab) don't include llama-cpp-python - the node installs it on first run, and you may still need to point it at a CUDA build if the auto-install grabs a CPU wheel. If you want guaranteed GPU acceleration, install it yourself first: pip install llama-cpp-python.

Where people get burned

Two classic traps. First, the auto-installer can leave you on a CPU-only wheel on Windows if your CUDA version doesn't match - the README targets CUDA 12.8 with prebuilt wheels, so a mismatched driver means slow inference and confusing errors. Second, model placement: GGUF files must live in models/text_encoders/ or models/clip/ or the dropdown won't see them. And remember the quantization math from the wider ecosystem - Q8 is basically fp16 at half the size, so if you can fit Q8, just use it. Don't drop to Q4 and wonder why your prompt optimizer got dumber.