通义千问VL 本地多功能视觉分析

The Qwen2VLLocalCaption node (display name: 通义千问VL 本地多功能视觉分析, "Qwen2-VL local multifunctional visual analysis") is the offline half of the pack's Qwen2VL pair from yiwangsimple/ComfyUI_GroqChat. No API, no key, no account - it runs the open Qwen2-VL-2B-Instruct model on your own GPU and returns its answer as text. Same five task modes as the API sibling (describe, OCR, visual reasoning, Chinese understanding, prompt generation), but free and unlimited.

The trade is what you'd expect: a 2B model is noticeably weaker than the cloud qwen-vl-max tier, and - the big one - this node does not auto-download anything. Unlike the pack's other local captioners, which pull their weights from Hugging Face on first run, this one looks for a model you have to place on disk yourself. Get that path right and it just works; get it wrong and it raises an error immediately.

How it works

On startup it searches three locations for the model, and the first that exists wins:

1. ComfyUI/models/prompt_generator/Qwen2-VL-2B-Instruct
2. models/prompt_generator/Qwen2-VL-2B-Instruct
3. Qwen2-VL-2B-Instruct (relative to the working dir)

It then loads Qwen2VLForConditionalGeneration via transformers with qwen_vl_utils, applies the chat template, and generates with sampling. The model stays resident in memory once loaded and only reloads if you change device or precision.

One code detail worth knowing: every input image gets resized to 224×224 with OpenCV before it's processed. That's a hard squash, so fine detail and small text suffer - if you're using the ocr task on anything with tiny fonts, expect the output to be rough.

Inputs and outputs

• image - the IMAGE tensor to analyze.
• prompt - your instruction (default is Chinese; English works fine).
• task - general, ocr, visual_reasoning, chinese_understanding, or prompt_generation.
• temperature (0–1, default 0.7), max_tokens (default 1024).
• device - cuda or cpu. CPU is technically an option; it's slow.
• precision - float32 or float16 (default float16, about half the VRAM).

Output is a single result STRING.

Installing it

The pack install is the usual:

cd ComfyUI/custom_nodes
git clone https://github.com/yiwangsimple/ComfyUI_GroqChat

Restart ComfyUI. Then the manual model step:

mkdir -p ComfyUI/models/prompt_generator
cd ComfyUI/models/prompt_generator
git clone https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct

Keep the folder name exact - Qwen2-VL-2B-Instruct. The pack's requirements include transformers, qwen-vl-utils, and opencv-python, though the pinned floor (transformers>=4.41.1) predates Qwen2-VL support; a fresh pip install resolves to a recent enough version, but if you have an old pinned transformers, upgrade it before debugging anything else.

Common issues

The dominant failure is the model path: the node raises "Could not find the Qwen2-VL model" if your folder is missing, misnamed, or empty - so verify the exact models/prompt_generator/Qwen2-VL-2B-Instruct location. After that, keep expectations calibrated for the 224×224 resize (OCR on small text is weak) and the 2B model's limits. If you need crisp detail extraction, the API node in the same pack is the better tool; use this one for cheap, private bulk captioning.