Nodes/ComfyUI_zura_nodes/Zura · Finish Speech Lip Sync
ComfyUI Node

Zura · Finish Speech Lip Sync

Paste the animated mouth back and walk away with a VIDEO

By ZURAVFX·Created about a month ago·Updated a day ago· 2
Zura · Finish Speech Lip Sync
  • images
  • speech_faces
  • face_boxes
  • reference_video
  • pose_data
  • VIDEO
◄fps24.00►
◄mouth_detail0.35►

The compose node is where the lip-sync stage stops being a pile of face crops and becomes a clip you can watch. It takes your generated frames, the animated faces, the boxes that say where each face lives, and the reference video to steal audio from - and hands back a finished VIDEO.

If you already have mouth motion from the previous node, this is the other half of a two-node finish: Zura · Mouth Motion from Audio makes the mouths, Zura · Finish Speech Lip Sync puts them back on the actors.

The inputs

  • images - your generated frames, the ones you want to keep. Everything outside the mouth stays exactly these pixels.
  • speech_faces - the batch from the mouth-motion node.
  • face_boxes - the per-frame detections. Yes, the type string really is BBOX, with a trailing comma; that's not a typo in this page, it's the native socket name from WanAnimatePreprocess and the pack matches it deliberately.
  • reference_video - the video the audio comes from. This is the one that trips people up (see below).
  • fps (default 24) - the timeline rate used to fit the audio.
  • Optional pose_data - pass the same detection payload you gave the mouth node. It's used to build a smarter mouth mask from the face landmarks rather than a fixed oval.
  • Optional mouth_detail (default 0.35, 0–1, step 0.05) - how much high-frequency mouth detail to recover when the crop is scaled down into place. Higher keeps more crispness and more risk; the default is a sensible middle.

One output: VIDEO, with your frames and the fitted audio already muxed.

What's happening inside

Four things, in order. The face crops are bicubic-resized back down to the box they came from. mouth_detail runs through a detail-recovery pass on the pasted region. A feathered mouth-finish mask is built for that box - from landmarks if pose_data is present, from the box geometry otherwise - so the paste blends into the original chin and cheeks instead of leaving a visible rectangle. Then it composites with original * (1 - mask) + face * mask, meaning anything outside the mask is bit-identical to your generated frame.

The audio handling is the quiet part: it takes the soundtrack from reference_video, slices or zero-pads it to exactly frames / fps seconds at its own sample rate, and muxes the result. So the audio is not an input socket. You don't wire a track in here - you tell it which video the track belongs to.

Install

Pack-level only; the mouth engine's models are the other node's problem.

cd ComfyUI/custom_nodes && git clone https://github.com/ZURAVFX/ComfyUI_zura_nodes
pip install -r ComfyUI_zura_nodes/requirements.txt

Or install Zura Nodes from ComfyUI Manager (publisher ZURAVFX) and restart. You'll also want ffmpeg/ffprobe on PATH, since assembling video is not something the pack does in pure Python.

Common issues

"Speech faces and detections must cover every generated frame." The three batches disagree on length. Almost always because you trimmed or batched somewhere in the middle: the face guide produced N crops for the frames it was given, and you're now feeding images from a different point in the graph. Keep one frame source for the whole finish branch.

"No usable face on frame 37. Lip sync needs a visible human face." The detection box for that frame is degenerate - a profile, a hand over the face, or a frame the detector simply dropped. There's no clever fix; either fix the shot or don't lip-sync those frames.

The video is silent, or has the wrong audio. Check reference_video. It's easy to leave it pointed at a placeholder clip while your real audio lives in a Load Audio node three nodes to the left - the node never reads an AUDIO socket, so that track is invisible to it.

Mouth looks pasted on. Drop mouth_detail toward 0.15–0.25 and make sure pose_data is connected; the landmark-driven mask is much better around the lower lip than the fallback geometry.

CategoryZura/Audio

Inputs (7)

NameTypeDefaultDescription
imagesIMAGE—
speech_facesIMAGE—
face_boxesBBOX,—
reference_videoVIDEO—
fpsFLOAT24.001–120—
pose_dataoptPOSEDATA—
mouth_detailoptFLOAT0.350–1—

Outputs (1)

NameTypeDescription
VIDEOVIDEO—