QWEN3-VL 视频理解

Video understanding is where cloud vision earns its keep. Running a video VLM locally means loading the whole video into VRAM, decoding frames, and burning a lot of GPU time on something that might just need a three-sentence summary. This node skips all of that: point it at a video, ask what's happening, and Qwen3-VL answers from Alibaba's cloud. Your local GPU never sees a frame.

It's the video sibling of QWEN3VL_Image in the ComfyUI_QWEN3VL_API pack, and it's designed to pair with the pack's LoadVideoFromFolder node - load a folder of clips, feed them here one at a time, and you've got a batch video captioning pipeline that costs per call instead of per GPU-hour.

How it works

The mechanism is the same OpenAI-compatible chat call as the rest of the pack, with one big difference: the video is base64-encoded in full and sent to DashScope as a data URL in the message. There's no frame sampling on your side, no length control - the whole file goes over the wire and DashScope does the video-specific processing (Qwen3-VL handles temporal alignment server-side).

That design choice is also the main gotcha. Base64 inflates the file by about a third, and the API has practical limits on what it'll accept. A short clip is fine. A long or high-bitrate file is exactly what the README warns about: oversize videos can fail or time out. Trim before you upload, not after. And note the cost - video analysis is priced per call and bigger/longer video generally means a bigger bill.

The node takes a video_path string as its main input, but it also accepts a video input (VIDEO type) that takes priority. The handling there is best-effort - it probes for the path in several common shapes (dict keys, attributes, string) because there's no single standard for what a VIDEO-typed output contains across custom packs. LoadVideoFromFolder's output works, but if you're wiring in some other pack's video output and it doesn't resolve, fall back to typing the path directly.

Inputs and output

• video_path - the file path, absolute and existing. If video is connected, it wins; otherwise this is what gets sent.
• video (optional) - a VIDEO-typed input, typically from LoadVideoFromFolder.
• model - qwen3-vl-flash / qwen3-vl-plus / qwen-vl-max, plus dated pinned variants. Flash is the sane default for a folder of clips.
• user_prompt - the instruction. "Describe what happens in this video" beats "summarize" when you want scene detail.
• seed - determinism; bump it to force a re-run (the node caches on seed, same as the rest of the pack).
• api_key (optional) - or the ini file.

Output is a single STRING (text), cleaned of the special marker tokens the model sometimes emits for box grounding.

Install and setup

No weights, no GPU download - just the client library:

cd ComfyUI/custom_nodes
git clone https://github.com/flywhale-666/ComfyUI_QWEN3VL_API
cd ComfyUI_QWEN3VL_API
pip install -r requirements.txt

Restart, and sort out a DashScope API key (in api_key.ini as DASHSCOPE_API_KEY=... or via the QWEN_APIKey node). The account setup is the one friction point, especially outside China where Alibaba cloud verification can be a hurdle - budget time for it the first time.

Troubleshooting

• "视频文件不存在" - the path isn't resolving. Double-check it's absolute and the file actually exists.
• Call times out or fails on a big file - that's the size limit. Shorten or compress the video, or lower the resolution, then retry.
• VIDEO input not resolving - the path probing is best-effort. Try the video_path field directly instead of fighting another pack's output format.
• Filtering applies - it's a cloud API with server-side content policy, so don't plan on using it for explicit content. That's what local abliterated models are for.

And the standing caveat for this whole pack: it's small and new, with no real community track record. The source is one readable Python file if you want to check exactly what gets sent before you point it at your own videos.