Protoface Sync Lipsync 3 · Lip Sync

Lip sync used to be a project: a Wav2Lip install, a face-detection pass, a fight with the mouth region for every identity, and output that looked like a 2019 deepfake. The hosted models moved that whole job behind an API call. Sync Lipsync 3 is one of them, and the Protoface node is two media inputs, one dropdown, and a bill.

The pitch in the model's own description is the part that matters: it matches lip motion to supplied audio "without a per-identity training step." Drop in a clip of a person, drop in a voice track, get the clip back with the mouth re-animated to the audio. Same face, new words.

What you're actually doing

Two assets in, one out, all remotely. This pack ships nothing local - it's a schema-generated front end over Protoface's API, so your clip and audio upload, a job runs on someone else's GPU, and the result downloads into ComfyUI as a native VIDEO. The KB's API-wrapper doc covers the general shape: per-call metering, inputs leaving your machine, model-side moderation you can't tune.

Here the meter is 14 credits per second of input video in the Protoface catalog, with a one-second minimum. A 30-second clip is roughly 420 credits. That's cheap for a job that used to eat an evening, and not cheap if you're iterating blindly on a two-minute video.

The mechanism, and the one setting that matters

video and audio are both media inputs with the pack's standard mode dropdown. Video and Audio modes take native ComfyUI tensors - so a Load Video and a native audio node (or an audio node from a TTS run in the same graph) wire straight in. References mode takes a Protoface Reuse Result Asset handle: if the clip came out of, say, an image-to-video node in this same graph, route its files output through that node and skip re-uploading megabytes you already sent.

Then there's sync_mode, which is the only real decision on the node. Video and audio rarely match in length, and this dropdown decides what to do about it. The options are bounce, loop, cut_off, silence, and remap, and the field's own tooltip points at Sync's Media Content Tips guide rather than trying to paraphrase each one. cut_off is the default and the sensible first choice: it ends at whichever stream runs out first, so you pay for a clip that isn't padded or stretched past your material. If your narration is longer than your footage, one of the extend modes is what you want; if the timing of the mouth must match a specific beat, remap time-stretches.

Practical order of operations: trim the video to the length you want before you send it. The meter counts input seconds, and trimming is the one lever that reduces the bill without reducing quality.

Controls and outputs

Beyond sync_mode, there are the pack's two bookkeeping fields. generation_id is the identity of the paid job: fixed means reuse or resume the recorded run, increment means buy a new generation, and it is entirely separate from any model seed. request_scope is an advanced, non-secret per-node scope the frontend assigns automatically; headless clients must supply a stable unique value or the node refuses to run.

Outputs are video (VIDEO list), files (the original download), and run id. Wire video into a preview or a save node, files into Protoface Save File if you want the untouched container on disk, and keep run id - Protoface Resume Run retrieves that job later without resubmitting, which is exactly what you want if the browser drops while a two-minute render is in flight.

Because "input seconds" is the pricing unit, also note that the pack enforces sizes and modalities on the way in. A mismatched reference modality fails loudly ("reference modality is unsupported by this model"), which is better than a silent bad job, but it does mean you can't sneak an animated GIF or a stereo trick through the audio socket.

Install

Manager → search Protoface → install protoface-nodes → restart → reload the browser. Or:

cd ComfyUI/custom_nodes
git clone --branch v0.1.0 --depth 1 https://github.com/protoface-ai/protoface-comfyui.git protoface
python -m pip install -r protoface/requirements.txt

ComfyUI 0.41.0+ and Python 3.10+. Both av and Pillow are in the requirements and both are load-bearing here - av decodes the returned video and re-encodes audio tensors on the way out. Then set the key from the Protoface button in the lower right (server installs: PROTOFACE_API_KEY or PROTOFACE_API_KEY_FILE).

Where it bites

Bad audio in, bad lip sync out. The model is matching mouth shapes to phonemes; a noisy track or a voice with heavy processing gives it nothing clean to match. Normalise and de-noise the audio before you spend credits.

Faces that aren't really faces. Profile shots, sunglasses, a hand across the mouth, extreme motion - this class of model degrades exactly where a human dubbing artist would struggle. Test one cheap 5-second clip before committing to a long one.

Re-entering the key. The dialog stores the key in server memory, not in the workflow. Restart ComfyUI and you're logged out; the node that worked last night fails with an auth error.

Interrupt ≠ refund. Cancellation is best effort. Check the run's status in your Protoface account if a clip feels missing.