API Qwen ImgOrVideo2Text

API Qwen ImgOrVideo2Text is the one Qwen node that does both image and video understanding from a single input socket. Feed it a single image and it describes the frame; feed it a batch of frames and it describes the motion. It's the reverse-prompting tool for the img2vid era - the thing you use when you want a written prompt of what's happening in a clip, not just what's in one still.

That video capability is the reason to pick this over the plain Image2Text node. Reverse-prompting video is a real workflow need: you've got a clip you like, you want a prompt that captures the action to feed an image-to-video model, and no single frame tells you that. Send the whole frame batch and Qwen-VL reads it as a video. Same DashScope key, same zero-VRAM API pattern, just a richer input.

How it works

Your image is encoded as a base64 PNG and sent as an image_url content part - the same as Image2Text. The difference is the optional video input, also typed IMAGE: in ComfyUI, "video" through this node means a batch of image tensors (frames), and the source builds a video content part out of every frame's base64 PNG. If video is provided it wins over image. Model names containing "omni" stream with modalities=["text"]; the others do a plain completion. Key: DASHSCOPE_API_KEY or the field.

The inputs that matter

• image (IMAGE, optional) - one still frame. Skip it if you're feeding video.
• video (IMAGE, optional) - a batch of frames. Any node that outputs a frame sequence works: a VHS/VHS-related loader, an image-to-video decode, or a batch of images stacked together. This is the input that makes the node worth installing.
• prompt (multiline) - what you want back. "Describe the action and camera movement in this video" is the natural ask.
• system_prompt (multiline) - optional guardrails for output format.
• model - qwen-vl-plus-latest (default, cheap), qwen-vl-max-latest (stronger), or the omni models (qwen-omni-turbo-latest, qwen2.5-omni-7b) which handle the multimodal heavy lifting.
• api_key - env var or field.

Output is a single text STRING.

Installing it

Part of ComfyUI_Prompt-All-In-One (billwuhao). ComfyUI Manager → search Prompt-All-In-One, or:

cd ComfyUI/custom_nodes
git clone https://github.com/billwuhao/ComfyUI_Prompt-All-In-One.git
cd ComfyUI_Prompt-All-In-One
pip install -r requirements.txt

Restart; it's under 🎤MW/MW-Prompt-All-In-One.

Where people get burned

The video input is a batch of IMAGE tensors - if you wire in a single image and expect motion description, you'll get a still-image caption, because that's what you gave it. Make sure your video loader actually emits a multi-frame batch. Frame count matters too: feeding 300 frames as 300 inline PNGs is a fat API call, so sample frames down to a manageable count (a VHS "frame to count" / sampler step, or just a shorter clip) before sending. Cost follows frame count directly, which is the quiet trap of this node - it's convenient, and the meter remembers. And the standing API caveat applies: your video leaves the machine, and vendor filtering applies to whatever you send.