QwenVL Advanced (GGUF)

This is the GGUF QwenVL node with the llama.cpp engine's real controls exposed - context length, GPU-layer offloading, batch size, the image token budget. It's the node you reach for when you're trying to run a Qwen-VL model that's technically too big for your card, and you need to hand-tune the memory tradeoffs to make it fit. If the plain GGUF node already runs your model fine, use that; these knobs are for when "fine" isn't happening and you need to negotiate with your VRAM.

Why these knobs exist

llama.cpp's whole trick is that it doesn't have to put the entire model on your GPU. It can split the model between VRAM and system RAM, layer by layer. That's how the community runs 30B-class vision models in as little as 3-5GB of VRAM - you keep some layers on the GPU, spill the rest to CPU, and accept that it's slower in exchange for running at all. This node is where you dial that split. The standard GGUF node makes the call for you; here you own it.

The inputs that matter

• gpu_layers (default -1) - the headline control. -1 means "put every layer on the GPU," which is fastest if it fits. When it doesn't, set a positive number to offload only that many layers to VRAM and push the rest to system RAM. This is the single lever that turns "won't load" into "loads and runs, slowly." Tune it down until the model fits.
• ctx (default 8192) - the context window in tokens. Bigger context handles longer prompts and more video frames but eats memory. Leave it at the default unless you actually need more; raising it is a common cause of surprise OOMs.
• image_max_tokens (default 4096) - how many tokens the image is allowed to expand into. Higher means the model sees the picture in more detail (better for reading fine text or small objects) at the cost of memory and speed; lower it to save both.
• n_batch (default 512) - prompt-processing batch size. Larger can be faster but uses more memory; the default is fine for most people.

The generation controls (temperature, top_p, top_k, repetition_penalty, frame_count for video) and the basics carry over from the other nodes - model_name (the same 12 Qwen3-VL GGUF quants), preset_prompt, custom_prompt, max_tokens (up to 4096 here), device, keep_model_loaded, seed. Feed it an image or video, read the text off the RESPONSE STRING output. Most of these you'll never touch; gpu_layers, ctx, and image_max_tokens are the three that actually move the needle.

How to install it

It's part of the main pack, and it has the same extra requirements as the standard GGUF node. ComfyUI Manager: search ComfyUI-QwenVL. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/1038lab/ComfyUI-QwenVL.git
cd ComfyUI-QwenVL
pip install -r requirements.txt

Then the two GGUF-specific steps: install a vision-capable llama-cpp-python (follow the pack's docs/LLAMA_CPP_PYTHON_VISION_INSTALL.md), and manually download the GGUF model file plus its mmproj into ComfyUI/models/llm/GGUF/. Restart ComfyUI.

Common issues

Out-of-memory is the whole reason this node exists, and also its main failure mode. If a model won't load with gpu_layers at -1, that's the signal to set a positive value and offload - start conservative and raise it until you're just under your VRAM ceiling. If it loads but generation is painfully slow, you've offloaded too much to CPU; nudge gpu_layers back up if there's room. A second sneaky OOM source is ctx and image_max_tokens set high "just in case" - both allocate memory whether you use it or not, so keep them at defaults unless a specific job needs more. And the base GGUF gotchas still apply: no vision if the llama-cpp-python build isn't the vision one, and no image understanding if the mmproj file is missing from the folder.