🖼️ Local Vision Model Loader (GGUF)

If you've ever tried to run a vision-language model locally, you know the horror: the model downloads fine, and then there's a second file - the mmproj - that has to match, and if the naming is off by one character, nothing works and the error message is useless. This node is the part of the ComfyUI-GGUF-VLM pack that makes that pain disappear. It loads a Qwen2.5-VL GGUF plus its vision projection file, auto-matches the mmproj by filename, and hands you a VISION_MODEL config ready for image analysis.

How it works

A vision model in the GGUF world is two files: the main model weights and a small projector file (*mmproj*.gguf) that bridges the vision encoder to the language model. This loader scans the same model folders as the text loader - ComfyUI/models/LLM/GGUF/, text_encoders, clip, and clip/gguf - and runs an MMProjFinder over the names. It tries a whole family of naming patterns: Qwen2.5-VL-7B-Instruct-Q8_0-mmproj-F16.gguf, the plain -mmproj.gguf variant, lowercase f16, and the format where the quant suffix is stripped (Qwen2.5-VL-7B-Instruct-mmproj-f16.gguf). Keep the two files in the same directory and it almost always just finds the match.

The inputs

• model - the dropdown of VL models the pack detected. Click the 🔄 refresh button if you just dropped a new file in.
• n_ctx - context window, default 8192 (max 32768 here, tighter than the text loader's cap). Vision tokens are expensive; a big image eats thousands of tokens, so give this some headroom for long video analysis.
• device - Auto / GPU / CPU, same as the text loader.
• mmproj_file (optional) - the manual escape hatch. If auto-matching fails on a weird filename, type the mmproj path yourself.

The single output is model (VISION_MODEL), which feeds the 🖼️ Local Image Analysis node - that's where you write the prompt and get text back.

Install and the one gotcha that matters

Installing is the same as the rest of the pack: clone, pip install -r requirements.txt, and crucially install llama-cpp-python with CUDA from abetlen's wheel index (see the README install steps - pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu126 for CUDA 12.6). Drop the model and its mmproj into models/LLM/GGUF/, refresh, connect.

Here's the part worth knowing before you hit the wall: local vision here means Qwen2.5-VL, and only Qwen2.5-VL. The README is explicit that the local GGUF vision path supports the Qwen2.5-VL series. If you load a Qwen3-VL GGUF and get nothing but confusing behavior, that's not a bug on your end - the pack's vision registry is built around the 2.5-VL family. Want Qwen3-VL? That's the Transformers loader's job instead. If you see "No models found" in the dropdown, it's almost always either the wrong directory or a file whose name the registry doesn't recognize as vision-type.