Qwen2VL反推提示词

You have an image and you want a prompt that would produce it - for img2img, for image-to-video, or for captions on a training set. Qwen2VL_prompt runs a real vision-language model, Qwen2.5-VL, entirely on your machine and writes that prompt for you. No cloud, no key, no pay-per-call, and no content filter deciding what it's allowed to describe.

It's the sibling of the pack's Qwen2_prompt node - same author, same pack (AIGCZero/ComfyUI-tools_zero), same auto-downloading local model approach - but with vision bolted on. It's also a member of the VLM-captioner family that this community leans on heavily, so it's worth knowing where it fits: heavy and accurate on the description end, where Florence-2 is fast and cheap. The one weakness every captioner shares, including this one, is multi-subject attribution - two people in a frame and the model will start mixing up who's wearing what. Trust the caption for the general scene, audit it for details that matter.

What it does

Drop an image into the image input, and pick a job from the prompt_type dropdown. There are ten, and they cover the useful ground: 图像分析 (image analysis), a detailed deep-read variant, 姿态分析 (pose analysis, which is told to describe only the subject's body and to skip the scene entirely), three video-description modes (multi-segment camera moves, fast cuts, and 首尾帧 for first/last-frame img2vid), 电影分镜 (a three-shot storyboard script), a kontext tagging mode, and 用户输入 to just use your own instruction. The multi-cam video templates are written in a Wan-flavored cinematic Chinese style, which tells you the pack's author built this for the local video scene first.

There's also an optional video_path string input. Give it a path to an mp4 and the node runs ffmpeg on it first, extracting a frame per second, then describes the motion across the clip - that's how you get "a woman walks left as the camera dollies right" instead of a still-image caption.

Model choices are the Qwen2.5-VL 3B and 7B Instruct plus their -bnb-4bit quantized versions, and SkyCaptioner-V1 - Skywork's uncensored captioning model, a small but real community name on reddit since spring 2025 and the one to pick if you caption NSFW data and don't want euphemisms. Default is Qwen2.5-VL-7B-Instruct-bnb-4bit, which is the right balance on a 12GB card; 3B is what you reach for when the 7B won't fit.

Output is a single STRING - wire it to a text encoder, a prompt enhancer, or a text-saving node in a batch.

How it works

Models auto-download from HuggingFace into ComfyUI/models/LLM/ on first use - again, a multi-gigabyte first run. Images are converted tensor-to-PIL and pushed through the qwen-vl-utils vision pipeline; video goes through a ffmpeg subprocess first, so ffmpeg must be on your PATH or the video path input fails. The same defensive VRAM handling as its sibling is here: it detects OOM, calls ComfyUI's /free endpoint, clears, retries once, and tells you to re-run if it still can't fit.

Installing it

ComfyUI Manager → search "ComfyUI-tools_zero", or:

cd ComfyUI/custom_nodes
git clone https://github.com/AIGCZero/ComfyUI-tools_zero

Restart after cloning. The pack installs transformers, bitsandbytes, and qwen-vl-utils>=0.0.11 - the last one is the vision-specific piece this node can't live without.

Where people get burned

Budget for the first-run download and for the VRAM math: the 7B VL model is heavier than the plain 7B LLM, so on 8GB cards you're realistically on the 3B or the bnb-4bit 7B with keep_model_loaded off so it frees itself before your sampler needs the card. Video input silently depends on ffmpeg being installed - if you feed a path and get nothing useful back, that's the first thing to check. And while the node waits on you is not a thing here (it's not interactive), the temperature slider shares the same cosmetic quirk as Qwen2_prompt: it's handed to decode, not to generate(), so seed is your real reproducibility lever. Also, for the Qwen2_prompt → Qwen2VL_prompt pairing people love - caption an image, then have the LLM reshape the caption into a structured prompt - you'll want the VRAM juggling tools this pack also ships: ReservedVRAMNode to hold headroom and Zero_CleanCache to unload the model between stages.