Video Captioning

If you're putting together a video LoRA dataset, the bottleneck is almost never the clips - it's the captions. You need a natural-language description of every clip, and doing that by hand for hundreds of clips is exactly the kind of soul-draining task that makes people quit before the first training run. This node is the fix: feed it the video frames, and Qwen2.5-VL writes the caption for you, inside ComfyUI, on your own GPU. No API key, no upload, nothing leaving your machine.

It's one half of a two-node dataset pipeline. VideoCaptioning writes the raw description; its sibling CaptionRefinement (same pack) cleans it up. They chain together nicely, and the output is a plain string you can dump to a .txt next to your clips for Wan, LTX, or whatever you're training.

How it works

The node is a port of cseti007's Qwen2.5-VL-Video-Captioning script, wrapped in a ComfyUI node. You feed it an images tensor - any batch of frames, typically from a video-loading node - and it converts each frame to a PIL image, resizes to fit within 224–448px, and hands the whole sequence to Qwen2.5-VL as a "video" message via qwen-vl-utils. The model reads motion, camera work, and scene transitions, not just a stack of stills.

Under the hood it's plain HuggingFace transformers with bitsandbytes. The model auto-downloads to ComfyUI/models/LLM/ on first run - no separate model install step. Up to 768 frames get processed per call, so a few minutes of footage at 12fps is fine. One grounded quirk: it doesn't crash on a bad frame, it returns the error as the caption string, so keep an eye on the output box.

The inputs that matter

• images - your video frames (ComfyUI IMAGE type). This is the whole point; feed it from your video loader.
• user_prompt - the long default prompt is genuinely good: it asks for character appearance, every movement, camera angles, transitions. Edit it if you want captions in a specific style, but run it unmodified first.
• model_name - defaults to Qwen/Qwen2.5-VL-7B-Instruct. Recent versions also offer ShotVL-7B and SkyCaptioner-V1, so it's not just Qwen variants. 1.5B exists if VRAM is tight; 72B only if you're swimming in it.
• quantization_type - 4-bit is the sensible default. 8-bit is slightly cleaner but noticeably hungrier.
• temperature - 0.3 default, low, which is what you want for dataset captions. Crank it if outputs feel stale.
• keep_model_loaded - off by default, which means the model offloads after each run and frees your VRAM for the next node. Leave it off unless you're captioning many clips in a row and want to skip reloads.

The output, caption, is a single STRING - wire it into a text save node to write dataset files.

Installation

Easiest path is ComfyUI Manager: search "DeZoomer" and install ComfyUI-DeZoomer-Nodes. Manual install is the standard clone:

cd ComfyUI/custom_nodes
git clone https://github.com/De-Zoomer/ComfyUI-DeZoomer-Nodes.git
cd ComfyUI-DeZoomer-Nodes
pip install -r requirements.txt

Then restart ComfyUI. On the portable Windows build, run that last pip through python_embeded\python.exe -m pip instead.

Where people get burned

The README's requirements list flash-attn>=2.3.0, and that's the classic trap. Flash attention is a compiled package; on Windows without a matching prebuilt wheel it tries to build from source and that needs Visual Studio build tools - a wall of C++ errors later, you're on a forum at midnight. If Manager's install trips on it, grab a prebuilt flash-attn wheel for your CUDA version first, then install the rest.

The README says "at least 16GB of GPU memory," which is conservative - a 4-bit 7B model fits in a fraction of that. What actually eats VRAM is attention over hundreds of frames. If you're running out, drop to the 1.5B model, and remember the first run downloads the full fp16 weights regardless of your 4-bit setting - that's a ~17GB download before you generate anything, so go make coffee.

One honest aside: yes, this is scriptable in a few lines outside ComfyUI, and people do say so. But if your whole dataset-prep flow already lives in ComfyUI, staying in-graph beats juggling a half-dozen Python scripts. It's a workflow choice, not a technical one.