🖼️ Vision Model Loader (Transformers)

Here's the thing the local GGUF vision loader won't tell you: it only speaks Qwen2.5-VL. If you want Qwen3-VL - which is noticeably better at OCR, long videos, and following complex instructions - the pack switches engines entirely, away from llama-cpp-python and over to HuggingFace Transformers. This node is that loader: it sets up a Qwen3-VL model through transformers, with a couple of genuinely useful options that the GGUF path doesn't have.

What it loads

The model dropdown is deliberately small: two options, both from the "Huihui" line - Huihui-Qwen3-VL-4B-Instruct-abliterated and the 8B variant. If you've been in the local LLM scene a while, Huihui needs no introduction: the prolific HF uploader whose "abliterated" quants remove the refusal training, so the model won't lecture you about content policy mid-caption. For image captioning that's usually a feature - you get straight descriptions instead of "I can't assist with that." The node pulls these via huggingface-hub (it's in the pack's requirements), so the first run downloads weights.

The inputs that do real work

• quantization - none, 4bit, or 8bit. This is bitsandbytes-style quantization through Transformers, not GGUF. If VRAM is tight, 4bit is the move; the KB's GGUF panel warns lower quants degrade - same rule applies here.
• attention - eager, sdpa, or flash_attention_2. The tooltip recommends flash attention for Qwen3-VL; it's faster, but it needs the flash-attn package installed and your GPU supported.
• keep_model_loaded - the useful one. Leave it off and the model unloads after inference, freeing VRAM for diffusion. Turn it on for repeated calls (like batch captioning) where reloading would waste more time than holding it.
• min_pixels / max_pixels - the resolution budget for images. Defaults are 200704 / 1003520 (roughly a 448×448 minimum up to a ~1024×1024-ish maximum). Bigger max_pixels = more detail seen, more VRAM and time. If captions are missing fine details, raise the max; if you're OOMing, lower it.

The output is model_config (TRANSFORMERS_MODEL), which feeds the 🖼️ Image/Video Analysis node - that's the multi-input analysis node, not the GGUF one.

Install

The pack install plus one extra note: because this path uses Transformers and flash_attention_2 is the recommended setting, transformers>=4.57.1 and accelerate come from requirements.txt automatically. pip install -r requirements.txt covers it. If you pick flash attention and get an import error, either pip install flash-attn (a notoriously slow compile on Windows) or switch to sdpa.

The honest tradeoff: this path is more flexible (Qwen3-VL, quantization, proper pixel control) but heavier - it pulls full HF weights instead of a single GGUF file, so downloads are bigger and first load is slower. That's the price of getting Qwen3-VL in ComfyUI, and for OCR-heavy or video-heavy captioning it's usually worth it.