Qwen-VL Model Loader

Most ComfyUI Qwen-VL loaders assume you have a big GPU and don't ask questions. This one actually cares about your VRAM. Qwen-VL Model Loader pulls a local Qwen-VL or Qwen3-VL checkpoint off disk, lets you pick a quantization level, an attention backend, and a device, and hands the result downstream as a QWENVL_MODEL passthrough that feeds the pack's LoRA loader and caption node.

It's also the most interesting node in the pack mechanically, because it borrows its loading internals wholesale from 1038lab's ComfyUI-QwenVL - the pack this one pairs with - including its quantization, FP8 handling, and SageAttention patching. If you've used that pack, this loader will feel familiar, minus the parts you don't need.

How it works

The loader looks for checkpoint folders in models/LLM/Qwen-VL/ (or anything registered under text_encoders in extra_model_paths.yaml) and exposes them in a dropdown. From there, the behavior branches on what you picked:

• 4-bit / 8-bit use bitsandbytes quantization configs (NF4 for 4-bit). These, plus any model with fp8 in its name, force SDPA attention - flash-attn is incompatible with quantized weights, so the loader overrides your choice for safety.
• FP8 models get a special path: loaded to CPU first with device_map=None, any meta tensors materialized (including sharded checkpoints via the safetensors index), then moved to your device. This is the fiddliest branch and the one 1038lab spent the most effort on.
• sage attention loads the model with SDPA first, then patches the attention layers afterwards using 1038lab's set_sage_attention(). That needs the upstream pack installed and sageattention. If either's missing, the loader falls back to SDPA with a console warning rather than crashing.
• auto (the default) just picks the best backend available: SageAttention if present, then flash-attn 2, then SDPA.
• use_compile wraps the model in torch.compile(mode="reduce-overhead") for extra throughput when CUDA's available. First run compiles and takes a while; if that annoys you, leave it off.

The output is a QWENVL_MODEL carrying {model, processor} - the processor is included, which is why the Caption node works with zero extra setup.

The inputs that matter

• quantization - 4-bit (VRAM-friendly) is the default and the right answer on anything ≤ 12 GB. 8-bit is the balanced middle. "None (FP16)" is for when you've got the headroom; FP16 at 8B parameters is ~16 GB just for the weights.
• model_name - your checkpoint folder. If the dropdown shows only "(none)", the folder isn't where the loader looks or lacks a config.json; restart ComfyUI after dropping a model in.
• attention_mode - leave on auto until you hit a specific problem. sage is the speed pick if you've installed the upstream pack plus SageAttention.
• device - auto is fine on a single GPU. The other values are there for CPU or Apple Silicon (MPS) runs.
• use_compile - off by default, on for a speed bump if you can stomach the compile wait.

Installing it

All three nodes install together. Manager search "ComfyUI Qwen-VL LoRA", or:

cd ComfyUI/custom_nodes
git clone https://github.com/Dangocan/comfyui_qwenvl_lora
cd comfyui_qwenvl_lora
pip install -r requirements.txt

Hard dependencies are just peft and transformers. The optional ones are where the trouble lives: bitsandbytes for 4/8-bit, flash-attn for flash attention 2, and sageattention plus the 1038lab ComfyUI-QwenVL pack for Sage mode. Skip the optional trio and everything still works - you'll just be on SDPA.

Where people get burned

The recurring theme is model folder shape. The loader wants full Qwen-VL checkpoints, not pruned text encoders - a lot of people try to point it at the Qwen3 TE their image model shipped with, and captioning silently fails or the dropdown stays empty. The README's example is a full Huihui Qwen3-VL-8B-Instruct-abliterated model; the "abliterated" part means the refusal behavior was trained out, which matters if you're captioning content the stock model would hedge on.

The FP8 path only triggers when the folder name contains fp8 - a normal model won't get it, and an FP8 file renamed without the tag will load badly. And remember the flash-attn conflict: if you explicitly pick flash_attention_2 on a 4-bit load, you're getting SDPA whether you like it or not. The console prints the actual backend used - when in doubt, read it.