🧠 QwenVL Unified (HF / GGUF)

What it actually does

This is the QwenVL node you'd reach for if you only want to learn one node from this pack. It's a vision-language model inside your graph - feed it an image or a video, get text back, and that text is usually a prompt. A local VLM reads your reference and writes the paragraph you feed into Wan, LTX, Z-Image or whatever is doing the rendering. That pattern is routine now; it's how most people stopped staring at a blank prompt box.

What makes this node interesting is that it's the merged one. The pack used to ship separate transformer and GGUF nodes, and in v2.8.0 they were folded into a single node with a backend dropdown. The old classes still load so existing workflows don't break, but if you're starting fresh, this is the one.

The mechanism, and the one thing that trips people up

The node is a dispatcher: it holds both the HF/Transformers implementation and the llama.cpp GGUF implementation and forwards your inputs to whichever matches. That's why HF and GGUF models share one dropdown.

Here's the trap - the model_name prefix decides the route, not the backend switch. Entries are prefixed HF: or GGUF: , and the code checks that prefix. The tooltip says it plainly: "keep them aligned." Pick backend: HF Transformers with a GGUF: model and you get the GGUF path anyway, plus a mystery about why a GGUF model is pulling weights through the Hugging Face loader. It's a cosmetic dropdown sitting next to a functional one.

Inputs worth setting

Three of them cover 90% of uses:

• preset_prompt - 27 built-in instructions. 🖼️ Detailed Description is the default and the right pick for captioning or img2img seeding. The video presets (🎬 MiniMax H3 NSFW, 🎥 Wan 2.2 NSFW I2V, the LTX 2.3 ones) are template-shaped: they tell the model to write shot-native prompt text rather than describe the picture. Match it to the model you're feeding.
• custom_prompt - your own text, and note it combines with the preset template rather than replacing it. That reversal of what most people expect is deliberate in this pack.
• seed - this is where reproducibility lives. Reuse the seed with keep_last_prompt and the node hands back the previous prompt instead of re-running the model.

The rest is plumbing. max_tokens defaults to 8192, which long video presets need. keep_model_loaded keeps the VLM resident so the second run skips the load. passthrough returns custom_prompt straight through without loading a model - handy while you're wiring a graph. camera_tag injects a movement tag like [PAN_LEFT] and re-appends it at the end of the prompt, because these models weight the end of a prompt more heavily.

Optional media inputs: image is your single reference, image2 the second one (the R2VA presets use a picture 1 / picture 2 pair), and video takes a batch of frames - frame_count (16 default, 64 max) sets how many get sampled, and applies only to video.

Outputs

One: RESPONSE, a plain string. Wire it into anything that eats text - an encoder's prompt field, a Save Text node, the prompt widget on a Wan or LTX sampler.

Install

ComfyUI Manager → search QwenVL-Mod: Enhanced Vision-Language. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/huchukato/ComfyUI-QwenVL-Mod
cd ComfyUI-QwenVL-Mod && pip install -r requirements.txt

Then restart. Models download on first use into ComfyUI/models/LLM/Qwen-VL/ (HF) or ComfyUI/models/LLM/GGUF/ (GGUF). Drop a .gguf plus its mmproj file in there, restart, and it shows up without any JSON editing - the v2.4 local-discovery behaviour that fixed the loudest complaint about this node family.

Two install notes that matter more than they look. requirements.txt pins transformers>=5.2.0 - an aggressive floor that can upset packs wanting something older. And the GGUF path needs llama-cpp-python built with vision support, which is not in requirements.txt; the pack ships a placeholder pointing at its own install guide. Missing-chat-handler errors come from there.

One habit: a custom node is arbitrary Python with your user's file access the moment ComfyUI imports it, and "vision node that downloads its own models" is exactly the shape of the one that shipped malware and ended in a prosecution. This pack is widely installed and open source, which is the bar.

When it misbehaves

Garbled output - raw tokens, mixed languages, model meta-text instead of a caption - is almost always a transformers version mismatch, not anything you set on the node. It's the most reported failure across this node family. Install this pack in a venv you can roll back, because it pins a newer transformers major version than most.

Suddenly slow - twenty seconds to three minutes after a torch upgrade usually means it fell back to CPU. Check GPU utilisation during a run before tuning anything.

Refusals - stock Qwen instruct models are censored, a fine-tuning artefact rather than a licence one. The pack's defaults are community "heretic"/abliterated builds for exactly that reason; swap in a stock Qwen3-VL-*-Instruct and you'll get lectures instead of captions.

And the honest ceiling on all of this: VLMs mix up who's wearing what when there are two people in frame. For LoRA dataset captioning, auto-caption a big set and hand-audit a small one.