Prompt Rewriter Options

The pack's main node, Prompt Rewriter, is deliberately dumb: it takes your prompt and a bag of options and does what it's told. All the actual decisions happen here, in its companion Options node. Connect this node's options output into the main node's options input and suddenly you control which LLM does the writing, how much of it lands on GPU versus CPU, whether it "thinks" before answering, and how creative it's allowed to get. Without it, the main node has no model to load at all - it's not optional, it's the brain.

The model list - the field that matters most

model is a dropdown that lists every .gguf file found in ComfyUI/models/LLM/gguf. The node creates that folder for you on first run, so it always has somewhere to look. No files, and you'll see a lonely "No models available." The tooltip mentions models marked with ⬇ downloading automatically - in the current code that list is disabled, so you download the GGUF yourself and drop it in. Projector files are filtered out of this list, so you never accidentally pick a vision projector as your text model.

GPU layers - the whole multi-GPU story in one string

Leave gpu_layers empty and everything goes to GPU 0. Otherwise it's a comma-separated recipe: gpu0:0.7 puts 70% of layers on GPU 0 and 30% on CPU; gpu0:0.5, gpu1:0.4 splits across two cards with the rest in system RAM. If you're running a 4B or 8B model on a single card, empty is exactly right and you can forget this field exists.

How the model behaves

• enable_thinking - on by default, meant for "thinking" models like Qwen3-Thinking. The model deliberates before answering, which usually gives richer rewrites at the cost of time and tokens.
• system_prompt - an override for the pack's carefully-worded default (the one that locks in your core elements and forbids "8K"/"masterpiece"-style filler). That default is genuinely good and prompt-specific; leave this empty unless you know you want different behavior.
• use_model_default_sampling - skip the sampling knobs below and trust the model's own defaults. If you have no strong opinion about temperature, this is a perfectly reasonable setting.

The sampling knobs

temperature (higher = more creative, lower = more literal), top_p, top_k, min_p, and repeat_penalty. Defaults are 0.8 / 0.95 / 40 / 0.05 / 1.0, and they're sensible for prompt rewriting - don't touch them until you've seen a few outputs and know which direction the model is off in.

The VRAM budget

context_size is the dial to watch. It's the total token budget for input + output, and higher values eat more VRAM - which matters when the LLM is sharing your card with a diffusion model. 4096 is the default and plenty for prompt rewriting. max_tokens caps the length of the rewrite; 4096 is generous. flash_attention should stay on unless your GPU misbehaves.

Vision inputs

image_1 through image_5 accept IMAGE tensors for vision-language models. Feed the model a reference image alongside your prompt and it can extend or describe what it sees. This needs a VLM GGUF plus its matching mmproj file in the same models folder (the README's naming convention: Qwen3-VL-4B-Thinking-mmproj-BF16.gguf), and the node auto-detects the matching projector from the model name.

The single output, options, goes into the Prompt Rewriter node's options input. That's the entire job - the node is a control panel, not a renderer.

Gotchas

Change the model and the node restarts the llama-server, which means a full model reload - the first run after a swap is slow, so don't read it as a hang. Connect images but have no mmproj file and the main node errors out with a message telling you exactly that. And if the dropdown shows "No models available," you've put the GGUF in the wrong place - it must be ComfyUI/models/LLM/gguf, not one of the other model folders. Install is the same as the main node: clone the repo, pip install its three requirements, and get a llama.cpp backend in place (Vulkan via winget install llama.cpp, or the bundled CUDA installer) before anything actually runs.