🧠 QwenVL Unified Advanced (HF / GGUF)

Same node, unscrewed

QwenVL_Unified_Advanced is the same unified HF/GGUF vision-language node as its sibling, with the panel opened up. The simple version hardcodes its sampling - temperature 0.6, top_p 0.9, no beam search, full GPU offload, 32K context, FP16 on the HF path. The Advanced version exposes all of that. Same dropdowns, same preset system, same RESPONSE output.

Reach for it when the simple node is nearly right: you want beam search because the node keeps writing something florid, or you need it to actually run on CPU, or you're on a 6 GB card and 4-bit isn't negotiable.

The knobs that matter

device - auto, cpu, mps, plus a cuda:N entry for every GPU ComfyUI can see. auto is right unless you're running a second card or deliberately offloading the VLM so it doesn't fight your diffusion model for VRAM. Note that CPU works but "works" is doing heavy lifting: a 9B VLM on CPU is a coffee break per prompt.

quantization - 4-bit (VRAM-friendly), 8-bit (Balanced), None (FP16), defaulting to FP16. This is the HF path only. The pack removed on-the-fly quantization from regular use after it caused OOM crashes on a 5090, and the house rule since is: HF nodes for quality, GGUF nodes for anything quantized. If you're at 8–16 GB and want quants, stop fighting this widget and pick a GGUF: model instead.

attention_mode - auto, sage, flash_attention_2, sdpa. auto walks Sage → FlashAttention 2 → SDPA in that order. On paper Sage is fastest; in practice this pack has added, removed and re-added SageAttention support across several versions, and the tooltip itself says SDPA is stable and recommended. If output gets weird or a run dies in the attention layer, force sdpa and re-test. It costs you some speed and buys back predictability.

ctx - context window, default 32768, up to 262144. GGUF path only. Video presets with long templates plus sampled frames are what eat this; if the model starts truncating or writing incoherent endings, raise it and watch your VRAM.

gpu_layers - -1 offloads everything to the GPU. Lower it when a GGUF model and a diffusion model can't share the card; layers that don't fit go to system RAM and it gets slower but it stops exploding.

image_max_tokens (4096) is the vision-side budget - how many tokens the image itself is allowed to consume - and frame_count (16, up to 64) is how many frames get sampled from the video input. Crank frame_count and you pay in context.

The sampling group (temperature, top_p, repetition_penalty, num_beams) behaves like every other text model: num_beams above 1 disables temperature and top_p and trades speed for steadiness. repetition_penalty defaults to 1 here, i.e. off. use_torch_compile needs a CUDA/torch build that supports it and a warm-up run before it pays off.

The structural gotcha

Every knob is listed whether or not your chosen backend uses it. Look at the label groupings in the source: quantization, attention_mode, use_torch_compile and num_beams are "HF-specific"; ctx, n_batch, gpu_layers, image_max_tokens, top_k and pool_size are "GGUF-specific." The model_name prefix picks the route, and the other side's settings are silently ignored. Set ctx to 131072, run an HF: model, and nothing happens - no warning, no error, just a setting that does nothing. Align backend and the prefix, then only tune the group that applies.

Inputs and outputs

Required: backend, model_name, preset_prompt (27 presets, 🖼️ Detailed Description by default), camera_tag, custom_prompt, device, max_tokens, the sampling floats, keep_model_loaded, seed, keep_last_prompt, passthrough, then the backend groups above. Optional: image, image2 (picture 1 and picture 2 for the R2VA presets), and video.

Output is a single RESPONSE string - same as the simple node, so you can swap between them without rewiring.

Install

Same pack, no special steps:

cd ComfyUI/custom_nodes
git clone https://github.com/huchukato/ComfyUI-QwenVL-Mod
cd ComfyUI-QwenVL-Mod && pip install -r requirements.txt

Manager route: search QwenVL-Mod: Enhanced Vision-Language. Restart afterwards. GGUF models need llama-cpp-python with vision support (Qwen3VLChatHandler), which is a separate build you have to install yourself and the most common reason the GGUF half of this node looks broken on a fresh setup.

Troubleshooting

If the node's answers degrade into raw tokens or system text, walk back the exotic settings first - attention_mode to sdpa, use_torch_compile off, num_beams back to 1 - then check your transformers version, which is where this node family's garbled-output reports usually end. If an HF: model is fine on one ComfyUI install and crawling on another after a torch/CUDA upgrade, it's probably not on the GPU at all; verify before tuning.

And keep one expectation straight: this node doesn't generate anything. It writes text upstream of your sampler, and the thing that decides whether your image is good is still the model you're prompting.