VLM Prompt Advanced (GGUF)

"VLM Prompt Advanced (GGUF)" is the plain "VLM Prompt (GGUF)" node with the whole llama.cpp tuning panel exposed. Same local vision-language model, same base64 frames, same prompt output - but instead of a handful of knobs you get the full sampler, the context window, GPU layer control, and device selection. You reach for it when the simple node works but you need it to fit or behave differently.

The trigger cases are pretty concrete. You're on a 8GB card and the default model OOMs during generation, so you want to pull layers off the GPU. You're prompting a longer video and need more frames sampled. Or you're looping the node and want it snappier. That's this node's whole reason to exist.

What the extra knobs actually do

• gpu_layers - how many model layers live on the GPU. Default -1 means all of them. Dropping it to 20–30 offloads work to CPU and trades speed for VRAM headroom. This is the one beginners end up touching.
• device - auto or cpu. Leave it on auto unless you have a reason.
• frame_count - how many evenly sampled frames from the video input get sent to the model (1–64, default 16). More frames = better motion description, slower run, more tokens.
• ctx - the context window in tokens (default 8192). Bump it for long outputs or long video analysis; the Qwen defaults in models.json are 16384.
• image_max_tokens - token budget for the vision side. Default 4096.
• temperature / top_p / top_k / repetition_penalty - the usual sampler dials. Defaults are sane: temperature 0.6, top_p 0.9, repetition penalty 1.2. Lower temperature for more consistent describing, raise it if the model keeps writing the same sentence.
• n_batch and pool_size - performance/threading settings. Genuinely leave these alone unless you know what you're doing.

Everything else matches the simple node: the same 4 GGUF models, the 27 system_prompts.json presets, system_prompt, custom_prompt (overrides the preset when filled), max_tokens, keep_model_loaded, and seed. The output is the single prompt STRING that feeds a Text Encode or a video model's prompt input.

Install and the one real gotcha

Install is the same as the whole pack - ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/mexxmillion/ComfyUI-VLMPrompt.git
pip install -r ComfyUI-VLMPrompt/requirements.txt

Then restart ComfyUI. Model files auto-download to ComfyUI/models/llm/GGUF on first use.

The gotcha is shared with the whole GGUF side of this pack: llama-cpp-python isn't in requirements.txt. You need a multimodal build with the Qwen/Gemma VLM chat handlers - the stock PyPI wheel may not include them. If you haven't hit the GGUF nodes yet, install a compatible build first, or these nodes just won't register. Model settings themselves live in models.json, which is data-driven: you can add a new model entry without touching Python, as long as the backend supports its architecture.

Common issues

• OOM during a video run - lower frame_count and/or gpu_layers before you touch the sampler.
• Repetitive output - nudge repetition_penalty up past 1.2 rather than cranking temperature.
• First run seems frozen - it's downloading the GGUF plus its mmproj file. Wait it out.