🧭 VLM Model Advisor

The most honest node in this pack: VLM Model Advisor admits that picking a vision model is a pain, and does the picking math for you. You tell it three things - what task you're doing, how much VRAM you have, and whether you're okay with remote services - and it returns a recommendation STRING telling you which backend and which model to start with. It's the pack's own opinionated guidance, surfaced as a node.

The recommendations aren't random. They're grounded in a model catalog (model_catalog_v2.yaml) that the pack ships, matched against your VRAM number and privacy preference. If the catalog's loaded, the recommendation even names the specific catalog entry with its recommended backend and minimum VRAM. That's genuinely useful when you're staring at a registry of GGUF quants and don't know if Qwen2.5-VL-3B or Qwen3-VL-8B is the right first download.

The three inputs

• task - the job: caption/标注, ocr/文字识别, prompt reverse/反推, json/结构化, video/视频理解, or custom/自定义. Each gets task-specific advice - video, for example, tells you to sample frames first with Video Frame Sampler, then feed a VLM.
• vram_gb - your GPU memory, default 8, up to 128. This is the big lever: the advice tiers by it (≥16GB → Qwen3-VL-8B GGUF Q4_K_M/Q8_0 locally; ≥8GB → Qwen2.5-VL-3B/7B Q4 or Qwen3-VL-8B Q4_K_M; below that → remote or CPU offload).
• privacy - 本地优先 (local-first), 远程可用 (remote OK), or remote preferred. Flips the recommendation between local GGUF and an OpenAI-compatible remote service.

What you get out

One recommendation STRING with the recommended backend, the model, and a task-specific suggestion. It's advisory, not magic: no model downloads, no config, no side effects. You read it, then go configure the matching loader or config node. Treat it as a smart starting point - the pack's authors' own defaults for your situation - rather than gospel, because the right answer also depends on your patience for downloads and your tolerance for quant artifacts.

Where it fits

The honest use is at the start of a workflow build, before you've committed to a backend. It's also a decent teaching tool: if you're wondering why a 3B model is the sensible local default on 8GB, the advisor's reasoning is basically the GGUF quality-ladder logic - Q4_K_M is the accepted compromise on constrained cards, Q8 when you have headroom. One nice touch: for OCR it points at Qwen3-VL and mentions InternVL as a remote/optional reference, which is genuinely current advice for text-heavy work.

The honest limits

It's a static rule engine reading a YAML catalog, not a benchmark of your actual machine. It doesn't know your CPU, your RAM, or that your 8GB card is shared with the diffusion model you're also running. The VRAM tiering assumes a dedicated GPU; if you're running the sampler and the VLM simultaneously, be more conservative than the advisor suggests. Also, the output mixes English and Chinese labels (the enum defaults are Chinese), so you'll want to know 本地优先 = local-first and 远程可用 = remote OK before you click.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/walke2019/ComfyUI-GGUF-VLM.git
cd ComfyUI-GGUF-VLM
pip install -r requirements.txt

Restart ComfyUI; it's under GGUF-VLM → 🚀 Business. It needs pyyaml (in requirements.txt) to read the catalog - one of the few nodes in the pack that has a real, non-model dependency.

Troubleshooting

If recommendation comes back without the "Model Catalog 2.0" line, the YAML failed to load - check that model_catalog_v2.yaml is still in the pack's root (a partial clone would break it). And remember it's advice, not a loader: if the advisor says Qwen3-VL-8B but your disk is full, that's a you problem, not a node bug.