QwenVL (GGUF)

Same idea as the standard QwenVL node - image in, description out - but running on the llama.cpp backend instead of Transformers. Why bother? VRAM. GGUF is the quantized single-file format that lets big models fit on small cards, and it's how people run Qwen-VL when the full Transformers stack won't fit or won't cooperate. If you've got a comfortable GPU and the regular QwenVL node already works, you don't need this. If you're on 8-12GB and fighting out-of-memory errors, this is the route that gets you there.

The tradeoff is upfront: GGUF nodes need a bit more setup than the standard ones. Once that's done, though, the payoff is real - the same trick that lets a 30B model run in 3-5GB of VRAM on llama.cpp is exactly what this node exposes.

How it works

GGUF is llama.cpp's quantized weight format - one file holds the model, compressed down a quality ladder (Q4, Q8, F16). The community consensus, well-established from the Flux era, is that Q8 is essentially indistinguishable from FP16 at half the size - if it fits, just use it - while Q4_K_M is the accepted compromise when you're truly tight on memory. Vision adds one wrinkle: the model file describes language, and a second file called an mmproj (multimodal projector) handles turning the image into tokens the model can read. You need both. That's why the pack asks you to install a vision-capable build of llama-cpp-python - a plain one can't see.

The inputs that matter

• model_name - 12 GGUF options: Qwen3-VL in 4B or 8B, Instruct or Thinking, each at F16, Q4_K_M, or Q8_0. Translation: pick your size (4B is the friendly default, 8B if you have room), pick Instruct for plain captioning, and pick your quant - Q8_0 for near-FP16 quality, Q4_K_M when you're squeezing, F16 if you somehow have the VRAM and want the GGUF path anyway.
• preset_prompt - the same nine jobs as the standard node: Tags, Simple/Detailed/Ultra Detailed Description, Cinematic, Detailed Analysis, Video Summary, Short Story, Prompt Refine & Expand.
• custom_prompt - overrides the preset with your own question.

Connect an image (or frames to video) and read the answer off the RESPONSE STRING output. Note what's not here compared to the Transformers node: there's no quantization or attention_mode dropdown, because with GGUF the quantization is already baked into the file you chose. max_tokens, keep_model_loaded, and seed round it out.

How to install it

The node ships in the main pack - ComfyUI Manager, search ComfyUI-QwenVL, or manual clone:

cd ComfyUI/custom_nodes
git clone https://github.com/1038lab/ComfyUI-QwenVL.git
cd ComfyUI-QwenVL
pip install -r requirements.txt

Two extra steps that the standard node doesn't need, and this is where people get stuck:

1. Install a vision-capable llama-cpp-python - the wheel has to provide the Qwen vision chat handler. The pack ships a guide for this at docs/LLAMA_CPP_PYTHON_VISION_INSTALL.md; follow it, because a generic llama-cpp-python install will load text but silently fail on images.
2. Download the GGUF files manually - the model file and its matching mmproj - into ComfyUI/models/llm/GGUF/. The README lists the HuggingFace repos (Qwen's official Qwen3-VL-*-Instruct-GGUF repos carry both the quants and the mmproj-*.gguf).

Then restart ComfyUI.

Common issues

Almost every problem here traces back to the two setup steps. If images come back ignored or the node errors on anything visual, your llama-cpp-python isn't the vision build - reinstall from the pack's guide. If the model loads but insists there's no image, you're missing the mmproj file or it isn't sitting next to the model in the GGUF folder; grab the mmproj-*.gguf from the same repo. If the model doesn't show up in the dropdown at all, check the folder path (ComfyUI/models/llm/GGUF/) and that the file actually finished downloading. And the ordinary VRAM math still applies: if an 8B Q8 is too much for your card, step down to 4B or to Q4_K_M rather than forcing it.