QwenVL (GGUF)

If the regular QwenVL node is the transformers route, "QwenVL (GGUF)" is the llama.cpp route: the same vision-language job - describe this image, caption these video frames, turn an image into a prompt - but running from quantized .gguf weights through llama-cpp-python instead of a full Hugging Face checkpoint. You reach for it when VRAM is the binding constraint, or when you already live in the GGUF world and want the captioner to share your disk and card the way quantized diffusion models do. It's the same pack, same install, different engine under the hood.

How it works

The node drives llama.cpp's OpenAI-compatible chat completion, and a vision-capable GGUF model is actually two files: the quantized language weights plus a separate multimodal projector (mmproj) that bridges images into the LLM. The pack ships a built-in catalog of Qwen3-VL GGUF models - the 4B and 8B, each in Instruct and Thinking editions, each in F16, Q8_0, and Q4_K_M - and auto-downloads both files from Qwen's Hugging Face repos into models/llm/GGUF on first use. Your image gets converted to a base64 PNG and handed to the model; video frames get sampled down to a set count first.

Model pick list, in short: Qwen3VL-4B-Instruct-F16/Q8_0/Q4_K_M.gguf, the same set for Thinking, and the same set again at 8B. The Q8 files are the "just use it" tier - near-FP16 quality at roughly half the size - and Q4_K_M is the tight-VRAM compromise. If a 4B in F16 is already fine on your card, the quant ladder here is about fitting more, not chasing quality.

The inputs that matter

• model_name - the GGUF variant; the pick list reflects what's in the catalog.
• preset_prompt - emoji-labelled jobs: 🖼️ Tags, Detailed Description, Ultra Detailed, 🎬 Cinematic Description, 📹 Video Summary, 🪄 Prompt Refine & Expand, and friends.
• custom_prompt - replaces the preset when filled.
• max_tokens - 512 default; captions are short, stories are not.
• keep_model_loaded - on by default so the second caption skips the reload.

Unlike the transformers nodes, this one takes up to three images (image, image_2, image_3) plus video - handy when you want the model to compare or reference multiple shots. The output is a single RESPONSE string, and here's the gotcha: this is not an output node, so nothing shows in the UI on its own. Wire RESPONSE into a ShowText node, the pack's aistudynow_SaveText, or a conditioning/CLIP encoder to actually use it.

Installing it

Same as every node in this pack - ComfyUI Manager, search "ComfyUI-QwenVL", or:

cd ComfyUI/custom_nodes
git clone https://github.com/aistudynow/ComfyUI-QwenVL

The catch specific to this variant is llama-cpp-python, which is in the pack's requirements but is the heavyweight of the lot: it compiles llama.cpp from source on install and needs a working C/C++ toolchain, so the first pip install can take a long while. On some setups you'll want a prebuilt wheel for your Python version instead. Don't skip it - without llama_cpp importable, the node fails with a clear "Install the GGUF vision dependency first" error.

Where people get burned

Three things bite. First, a model entry without a matching mmproj file means the node runs but ignores your images - it warns in the console and gives you a text-only answer, which is confusing the first time. Check that both the .gguf and its mmproj-* file landed in models/llm/GGUF. Second, the build pain above; if Manager's install of llama-cpp-python fails, install it manually with a prebuilt wheel before blaming the node. Third, VRAM math still applies - a Q8 8B captioner plus your diffusion model is two budgets on one card, so keep_model_loaded becomes a real choice rather than a free default. The community asked for GGUF support in this pack almost immediately after launch, and the author shipped it; it's the sensible low-VRAM path, not a compromise you'll only take when desperate.