Qwen3 VL Caption (Inverse Prompt)

If you've ever stared at someone else's image and thought "I wish I knew what prompt made this," this node is the answer. Qwen3 VL Caption (Inverse Prompt) runs a Qwen3-VL vision-language model locally in ComfyUI and turns whatever you feed it back into text - a prompt you can hand to a text-to-image model, a detailed caption for a training set, or just a description. No API key, no cloud call, no per-image bill. The name says "inverse prompt" and that's exactly what it does: prompt inversion, the reverse of the diffusion process you normally run.

It lives in the WingeD123/ComfyUI_QwenVL_PromptCaption pack, which is a family of caption nodes built around the same idea. This one is the flagship - the pack's most-searched node, and the reason most people install it.

How it works

Under the hood it loads a Qwen3-VL model through Hugging Face's transformers library (the source uses Qwen3VLForConditionalGeneration) and runs an image-to-text generation with a prompt-engineering instruction as the text input. The instruction asks the model to act like an expert prompt engineer and output a single detailed prompt for a text-to-image model. You can override that instruction entirely via the instruction input - useful when you want captions in a specific format, or a plain description instead of a prompt.

The pack ships a prompts.txt you can edit to change the default phrasing per language, and the node picks the last prompt matching your selected language. Leave instruction blank to use those defaults; fill it in and your text wins.

The inputs that matter

• model_path - a dropdown listing whatever sits in your ComfyUI/models/text_encoders folder. The pack doesn't auto-download anything, so this is empty until you add a model. See install below.
• lang - 中文, English, or bbox. The bbox mode is the fun one: it asks Qwen to output bounding-box coordinates for objects instead of a prompt, which you then feed into the pack's StringToBbox nodes.
• dtype - auto, 4bit, or 8bit. 4bit is the sweet spot on 6-8GB cards; the README's rule of thumb is 4bit under 10GB, 8bit up to 16GB, bf16 above.
• keep_model_loaded - leave True if you're captioning several images in a row; the model stays in VRAM instead of being torn down after every run.
• unload_other_models - defaults True and is a lifesaver: before loading Qwen it unloads everything else ComfyUI has in VRAM, which is what stops the classic "couldn't fit the caption model next to my checkpoint" failure.
• seed and max_side - seed seeds generation; max_side pre-scales the image's long edge (bigger = slower, and more VRAM). Default 512 is a safe place to start.

The image input is actually optional, and the node is happy to run on text alone - which sounds like a bug until you realize you can use it as a general-purpose Qwen3 chat node. If image is a multi-frame tensor (a video batch), set video_fps and it treats the frames as a video clip. Output is a single text STRING.

Install and model setup

ComfyUI Manager is the easy path - search "ComfyUI_QwenVL_PromptCaption" and hit install. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/WingeD123/ComfyUI_QwenVL_PromptCaption

then restart ComfyUI. The real dependency cost is transformers>=5.2.0, accelerate, peft, and bitsandbytes (that last one is what actually quantizes to 4/8bit). Then the model. The README's trick to avoid re-downloading 16GB: ComfyUI's own Qwen Image/Edit model already ships qwen_2.5_vl_7b.safetensors in text_encoders. Create a subfolder there, rename that file to model.safetensors, drop in the config files from Qwen's Hugging Face repo, and the node finds it. For Qwen3, download Qwen3-VL-4B-Instruct or Qwen3-VL-8B-Instruct and drop the folder into text_encoders.

Troubleshooting

Most first-run pain is one thing: an empty model_path dropdown because the model isn't in text_encoders. The node returns a plain "Failed to load model" string instead of an error, so if your caption comes back as that, check the path. The other classic is a stale transformers - Qwen3-VL needs a recent one, so let Manager update dependencies. If you're on 8GB, 4bit + keep_model_loaded True + max_side around 512 is the configuration people actually run.