🖼️ V2 Local Vision Model Loader

The whole point of this pack is that you don't need a server to run a vision model. V2 Local Vision Model Loader is the local-GGUF path: it loads a quantized vision language model straight into llama-cpp-python inside ComfyUI, hands it to the analysis nodes as a VISION_MODEL, and never talks to a port. If your machine has the VRAM and you want everything self-contained, this is your loader.

The dropdown is genuinely smart about what's on disk. Models are listed with a status prefix: ✓ means the file is already in your models folder, ✗ means it's not downloaded yet - and if you pick a ✗ model anyway, the node will download it from HuggingFace for you on load, including the matching mmproj file. The --- entries are just section headers (image models vs. video models) - pick one and you get an error asking you to choose a real model.

What it loads and where from

It scans ComfyUI/models/LLM/GGUF/ plus text_encoders, clip, clip/gguf, and VLM_GGUF for .gguf files, then matches them against the pack's model registry to figure out what each one is. The registry leans heavily on Qwen and - a thing you should know going in - includes abliterated/NSFW-caption variants (Qwen2.5-VL-7B-NSFW-Caption-V3-abliterated and friends). That's a feature for some people and a surprise for others; it's in the registry by design.

The inputs that matter

• model - the dropdown. The defaults shown are the pack's own registry picks, led by Qwen3-VL-8B at Q4_K_M (the practical default) and Q8_0 (better, bigger).
• n_ctx - context window, default 8192, up to 32768 here. Bigger context = more frames/images per call, but more VRAM.
• device - Auto (smart GPU-layer offload), GPU (all layers on GPU), or CPU. Auto is a good default; if you're fighting VRAM, Auto's heuristic is the first thing to tweak.
• mmproj_file - optional manual override for the multimodal projector. Normally the loader finds the right mmproj automatically; you only set this if auto-detection fails (mismatched names, custom setups).

The mmproj thing, since it will bite you

A vision GGUF isn't one file - it's the model plus a separate multimodal projector (mmproj) that maps images into token space. Mismatched mmproj is the classic "the model loads but can't see images" failure in the llama.cpp world. This pack's auto-download handles the matching when it fetches from HuggingFace, and the finder handles the common cases on disk, but if you manually drop GGUF files from HuggingFace, make sure the mmproj matches the model family or you'll get exactly that silent no-vision problem.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/walke2019/ComfyUI-GGUF-VLM.git
cd ComfyUI-GGUF-VLM
pip install -r requirements.txt

The llama-cpp-python>=0.3.16 in requirements.txt is the one that matters here - and whether it has CUDA support depends on how it built. On Windows the standard wheel usually works; if you're on Linux with a CUDA GPU and it's CPU-only, you'll want to rebuild it with CUDA for anything beyond 3B models.

Troubleshooting

Picking a --- group header errors - that's expected. If the loader can't find a model you just downloaded, you put it in the wrong folder (LLM/GGUF, not your checkpoint dir) or the list needs a refresh. If the model loads but analysis comes back gibberish or claims no image is visible, it's the mmproj. And the registry's auto-download will happily pull a 5GB file - check the model's size before you click if you're on metered bandwidth.