Gemini Omni Reference to Video

Keeping one character recognisable is the oldest unsolved gripe in this hobby. You generate someone great, then the next run gives you their cousin. The community's answer has drifted from adapters and character LoRAs toward a simpler trick: put the reference image in front of the model and let it look while it works. This node is that trick, applied to video - one or more reference images plus a prompt, and you get a 3–10 second clip at 24 fps featuring those subjects.

No checkpoint, no VRAM, no training run. Also no local option - Omni Flash is closed and API-only, so there's nothing to download. Treat it as the last stage of a pipeline: make your character or product image however you like, then put that exact thing into motion.

What actually happens on a run

One call, no polling. Veo in the same pack queues a long job and polls; Omni goes through Google's Interactions API and returns the video directly. The pack gives the call a floor of fifteen minutes before it gives up, since a 1080p render can outlast a default timeout - and you don't want to eat that bill.

It converts each connected image to PNG, sends them as image parts in batch order, then your prompt, and asks for the reference_to_video task at your chosen resolution.

The inputs that matter

reference_images is the whole point, and it takes an IMAGE. Several references means several images in the batch - wire a core Batch Images node in front of it if you need to combine them. Order is preserved, so batch slot 0 is <IMAGE_REF_0>.

prompt is where most people trip. You cite references by position using those angle-bracket tokens, in order, and that's the model's only handle on which picture is which:

The woman <IMAGE_REF_0> walks through the market at dusk, warm lamp light,
handheld camera drift. Her red coat <IMAGE_REF_1> is the only saturated colour.

A prompt that never names them leaves the model guessing which image is which. And there's no negative prompt field here - Omni Flash rejects them - so exclusions go inline: "no text on screen, no logo".

Then two easy ones: aspect_ratio gives you 16:9 or 9:16 and nothing else, and resolution is 360p, 720p (default), 1080p or 4k. 1080p and 4k are upscaled by Google rather than rendered, and because they blow past the inline size limit they come back as a Files API download - which the node waits on, then deletes. output_directory defaults to your ComfyUI output folder.

seed is the one that fools people. It is not a generation seed; Omni Flash takes no API seed, so this value is never sent. It only controls ComfyUI's cache. Leave it at -1 and every queue is a fresh, freshly-billed render. Pin it and ComfyUI hands back the video you already paid for.

client is optional - with a key in the pack settings you can skip the Gemini API Config node.

Output

video_path - a plain STRING pointing at the saved .mp4. Not an IMAGE, not a VIDEO tensor, so what you wire it into matters. The same pack's Preview Anything node recognises paths under your output folder and plays them inline with a download button, which is the fastest way to actually look at what you got.

Installing it

ComfyUI Manager is the sane route: search erpk, install ERPK Custom Nodes, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/eRepublik-Labs/comfyui-nodes-erpk.git erpk
cd erpk
pip install -r gemini/requirements.txt

That installs google-genai>=2.6.0 and Pillow. The SDK floor isn't decoration - the pack's own notes say 2.2.0 serialises aspect_ratio in a shape the API rejects, so if Omni calls fail with a weird 400 while the rest of the pack works, upgrade the SDK first.

Get a key from Google AI Studio and put it in Settings > ERPK > API Keys (or right-click the canvas > ERPK Settings). Keys stored there live in your user settings rather than the workflow, so sharing a graph doesn't leak them.

Where people get burned

The refusals are Google's, not the node's. No weights, no way to patch the filter - if Omni Flash won't make it, the node won't either.

Video is where an API budget disappears - every generation is a metered call, which is exactly why that seed widget matters more than it looks.

Two shapes of "it did nothing": an empty prompt raises Prompt cannot be empty, and no connected images raises Connect at least one reference image. On the version front, this node targets gemini-omni-1.1-flash - the earlier preview ID was retired on 2026-09-30, so an old install stops working and an update is the fix.

Set expectations on content, too. Omni is at its best restaging something already in front of it - a character, a product, a style - and shakier at inventing a big new element from nothing while holding a precise camera move. Reference-to-video plays to the strength. For the other job, you want a different tool.