Protoface Veo 3.1 · Reference to Video

Not a first frame - a cast list

Image-to-video takes one frame and animates from it. That's fine until you want the character to do something the frame doesn't support: turn around, appear from a different angle, stand in a room you haven't rendered yet. One frame can't tell the model what the back of her head looks like.

Reference-to-video is the different idea. You supply a set of images as references - a face, an outfit, a product, a location - and the prompt stages a scene using them. The frames aren't keyframes; they're the ingredients. It's the video version of what instruction-editing did to character consistency on the image side: stop trying to encode an identity, hand the model the actual pixels and let it look.

This node is Google's version on Veo 3.1, in Protoface - the beta pack that runs hosted models as native ComfyUI nodes, paid per generation through your Protoface account.

How it works

One shared adapter serves every node in this pack: your widgets become a schema-validated request, media is uploaded (content-hashed and cached, so reruns don't re-upload), the intent is journaled to SQLite, the job is submitted and polled, and the result is materialised as a native VIDEO.

images is the interesting input: a media input in native Image mode that grows as you connect assets, so three reference images means three connections, and input batches count against the model's reference ceiling. There's also a References mode for passing existing Protoface asset ids or reused result assets - that's how you feed it output from another remote job, like a character sheet you generated ten minutes ago in Nano Banana, without touching disk.

generation_id remains the paid-request identity, separate from seed. Fixed → reuse/resume. Incremented → new charge.

The inputs that matter

images - the cast. Keep it tight and intentional: one clear face beats five inconsistent ones. If the references disagree with each other, the model will average them, and averaged faces look like nobody.

prompt - describe the scene and action while naming which reference plays which part, so the model isn't guessing. This is the field where reference-to-video either sings or produces soup.

resolution defaults to 1080p, and generate_audio defaults on. There's no duration or aspect-ratio control exposed here - the model picks from the scene and the references. Set your expectations (and your downstream crop) accordingly.

negative_prompt and seed are media-style controls: Default omits the field entirely, Set supplies a value, Null sends an explicit null where the API allows it. Blank text and omission are distinct - a distinction the pack is unusually careful about, and one that matters if you're debugging why a "default" run looks different from a hand-set one.

Outputs

video (native VIDEO, list), files (PROTOFACE_FILE list of originals - the place to look for the untouched download), run id (string). Wire video into a save node, or into Extend to push the scene longer. Resume Run with the run id if the job outlives your browser.

Install

Manager → search Protoface → install protoface-nodes → restart → reload. Manual, from your ComfyUI directory:

git clone --branch v0.1.0 --depth 1 https://github.com/protoface-ai/protoface-comfyui.git custom_nodes/protoface
python -m pip install -r custom_nodes/protoface/requirements.txt

ComfyUI 0.41.0+ and Python 3.10+; use the install's own interpreter. av handles the native video side, Pillow the image side - both are in requirements.txt and both are cheap. Then click Protoface in the lower-right and enter your key.

Where people get burned

Reference caps, silently hit by batches. Connect a batch of six and you've sent six references. The pack errors clearly ("too many references, including items in input batches") rather than quietly dropping some, but only if you read the message.

Assuming references guarantee identity. They help enormously and they are not a face-lock. If a character must be identical across a dozen clips, this plus a LoRA trained on the final look, or a local identity workflow, is the more reliable production answer.

Rerunning the same graph expecting a new video. Fixed generation_id + unchanged inputs = the cached job. Increment to spend.

Content policy shock. Everything - prompts, references, results - goes to Google and is moderated by Google. That's the category tradeoff, not a bug in the node, and there's no local bypass for a closed model.