FL Gemini Video Captioner

Most captioning nodes look at a single frame. This one watches the whole clip. FL_GeminiVideoCaptioner hands your video (or a sequence of frames) to Google's Gemini API and asks it to describe what happens across time - the actions, the subjects, the setting, the vibe - not just what's in one still. If you're building a caption dataset for video LoRA training, or you need a text description of a generated clip to feed back into another model, this is the tool that understands motion.

The trade is the same as any API node: it's Gemini doing the work in Google's cloud, so you need a Gemini API key with billing enabled, and the captions are only as good as the model you pick.

How it works

You give it a video (via a video_path, or a batch of frames through the image input) and a prompt telling it what kind of caption you want. The node samples frames at your chosen rate, converts the clip to a WebM the API accepts, and sends it up to Gemini. Because Gemini is genuinely multimodal, it reasons over the sampled frames together - so it can say "a woman turns and walks toward the camera" rather than just listing what's in frame 1. You can even ask it to consider the audio track. Out comes the caption text plus one sampled frame so you can see what it looked at.

The inputs that matter

• api_key - your Google Gemini key. Required, and it needs billing.
• prompt - the instruction. The default asks for a detailed scene description; rewrite it if you want a specific caption style (short tags, a training-style caption, whatever your dataset needs).
• model - which Gemini to use, from gemini-2.5-flash (the default, fast and cheap) up to the Pro tiers. Flash is the right default for bulk captioning; reach for Pro only when the descriptions need to be sharper.
• frames_per_second - how densely it samples the clip. More frames = better temporal understanding but more tokens (more cost, slower). 1 fps is a sane start.
• max_duration_minutes - a ceiling so a long video doesn't blow up your token bill.
• process_audio - true if you want Gemini to factor in the soundtrack.

Outputs are caption (the STRING description) and sampled_frame (an IMAGE preview of what it sampled). The caption wires straight into a save-text node or onward into another prompt.

Installing it

Comes in the Fill-Nodes pack. ComfyUI Manager: search ComfyUI_Fill-Nodes, install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/filliptm/ComfyUI_Fill-Nodes

then restart. The pack shows a "Machine Delusions" splash in the console on load - harmless.

Where people get tripped up

• It's a paid API call. Nothing runs on your GPU. frames_per_second and max_duration_minutes are your cost dials - captioning a long clip at high fps sends a lot of tokens. Start at 1 fps and only raise it if the captions miss motion.
• Key and billing errors are the usual failure. If you get nothing back, it's almost always the key, quota, or a region restriction on the model - not the node.
• It converts to WebM for the API. That step needs the video encoding libraries the pack installs; if conversion fails on first run, it's a dependency issue, not the caption logic.
• Match the model to the job. Flash is cheap and fine for bulk dataset work. If you're captioning a handful of hero clips and want nuance, spend on a Pro model - but don't run a thousand clips through Pro out of habit.