AQ_QwenLoader

The setup half of the local-LLM pair. It downloads (if needed), loads, and keeps a Qwen2.5-VL model in memory, handing it to AQ_Qwen as a qwen_model object. One loader can feed several AQ_Qwen nodes, and the model stays resident - so the download and load cost is paid once per session, not per run.

What you're choosing between

Qwen2.5-VL is a vision-language model: it reads images, answers questions about them, and follows instructions. The dropdown offers unsloth's 4-bit quantized builds of three sizes:

• 3B - ~2GB VRAM, runs on nearly anything, fine for captioning and simple prompting.
• 7B - the sensible default; noticeably better reasoning for roughly 4–5GB VRAM.
• 32B - the "I have a big GPU" option; ~20GB+, and you need to actually want that much quality.

4-bit nf4 quantization is the sweet spot for all of them - the quality gap to full precision is small and the VRAM saving is huge.

How it works

On first load it downloads the model to ComfyUI/models/VLM (folder_paths.models_dir/VLM) via huggingface_hub's snapshot_download - a one-time multi-GB download. Then it builds a BitsAndBytesConfig (4-bit nf4, fp16 compute) and calls from_pretrained with your attention implementation and device choices. The qwen_model output is a small dict holding the model, the processor, and the model path - only meaningful to AQ_Qwen.

Inputs

• model - the three unsloth builds above.
• quantization - none or 4bit (default 4bit).
• attention - flash_attention_2, sdpa (default), or eager.
• device - auto, cuda:0, or cuda:1.

Output

qwen_model (QWEN_MODEL) → wire into AQ_Qwen.

Install

Part of AQnodes:

cd ComfyUI/custom_nodes
git clone https://github.com/2frames/ComfyUI-AQnodes
cd ComfyUI-AQnodes
pip install -r requirements.txt

or search "AQnodes" in ComfyUI Manager and restart. The requirements pull in transformers>=4.51.3, accelerate, qwen-vl-utils, and qwen-vl-utils[decord].

Gotchas

First run looks like ComfyUI froze - it's downloading several gigabytes; give it time. Keep quantization on 4bit unless you know you have VRAM to burn: "none" loads full fp16, and that's a lot on the 7B, let alone the 32B. flash_attention_2 requires flash-attn compiled for your CUDA, which is one of the most painful installs in the ecosystem - if you haven't done it, use sdpa (the default) or eager. And the model stays in VRAM after loading; you free it by clearing the graph or restarting ComfyUI, not by disconnecting the loader.