AV LatentSync1.5 Node

You've got a clip of someone talking and you want them to say something else. Different words, a different language, a voice that isn't theirs. The eyes, the head movement, the background - everything else stays exactly as shot. That's the job this node does, and it does it well enough that people have gotten genuinely uneasy about the results.

AV LatentSync1.5 Node is a ComfyUI wrapper around ByteDance's LatentSync 1.5, an audio-conditioned lip-sync model. You feed it a batch of frames and an audio track; it re-renders only the mouth region to match the speech. No driving video needed, no re-generating the whole clip. That's why LatentSync became one of the quiet workhorses of the open stack - the ByteDance releases everyone runs under their own names. LatentSync is the audio-conditioned specialist, distinct from the video models that generate audio natively. It exists for one specific job: dubbing and re-voicing footage you already have.

How it works

Under the hood the node is doing a little temp-file ballet. It writes your frames to a throwaway MP4 at 25 fps, resamples your audio to 16 kHz, then hands both to LatentSync's own inference pipeline: a Whisper model turns the audio into embeddings, the UNet denoises the frames conditioned on those embeddings, and a face mask confines the edit to the mouth so the rest of the frame isn't touched. SyncNet runs in the loop as a lip-sync sanity check. When it's done you get back a fresh set of frames - not the input - plus your audio resampled to 16 kHz.

The inputs that matter

• images - your talking-head frames, straight from something like VHS_LoadVideo (or from a generated clip).
• audio - the speech track. It should be clean speech; this is a lip-sync model, not a sound-effects machine.
• lips_expression - the one you'll actually fiddle with. Default 1.5; it maps onto LatentSync's guidance scale. Higher (2.0–2.5) means more pronounced mouth movement, good for expressive speech; lower means subtler movement for calm conversation. If the mouth looks over-animated, drop it. If it looks half-asleep, raise it.
• inference_steps - default 20. Push 30–50 for final renders, drop to 10–15 for quick previews. Classic diffusion trade.
• seed - LatentSync has a bit of stochasticity; same seed, same result.

Outputs are images and audio, and you wire both into a video encoder - the pack's own workflow uses VHS_VideoCombine to mux them back into an MP4.

Installing it

Install via ComfyUI Manager (search "ComfyUI-AV-LatentSync") or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/avenstack/ComfyUI-AV-LatentSync.git
cd ComfyUI-AV-LatentSync
pip install -r requirements.txt

Then restart ComfyUI. Two things will bite you on the way in:

1. FFmpeg must be on your system PATH. The node hard-fails with a clear error if it isn't. On Windows that means adding the bin folder of your FFmpeg build to PATH, not just installing the ffmpeg-python package.
2. The model doesn't auto-download. Grab ByteDance/LatentSync-1.5 from Hugging Face and drop it in ComfyUI/models/lipsync/latentsync, keeping the repo's structure - latentsync_unet.pt, config.json, stable_syncnet.pt, plus the auxiliary/, sd-vae-ft-mse/ and whisper/tiny.pt subfolders. It's several GB; this is not a five-minute install.

Also expect the first run to spend a while pip-installing its dependency stack (diffusers, transformers, mediapipe, face-alignment, decord, omegaconf and friends) and downloading face-detector support files. That's normal, not a hang.

What bites people

• VRAM. LatentSync 1.5 is hungry. Community reports for smooth runs sit around 20 GB; the node does adapt - it picks batch size and mixed precision from your VRAM, and works on 8 GB cards at reduced batch - but don't expect speed there.
• Only front-facing, fully visible faces. Profile shots, hands covering the mouth, quick cuts where the face leaves frame - all bad. Anime and cartoon faces don't work at all.
• The 25 fps assumption. LatentSync was trained at 25 fps. Feed it 60 fps source and things get weird; this node assumes your frames are already at a sensible rate, which is exactly what its sibling node (AVVideoLengthAdjuster) is for.
• Longer clips get twitchy. Community reports of mouth micro-jitter on clips past ~30 seconds, and the output tends to be softer than the source, so a face-enhance or upscale pass after lip-sync is common.
• Mouth movement during silence. If your audio has gaps, keep them clean or the model invents lip motion. The silent_padding_sec on the length adjuster exists partly for this.

The honest take: LatentSync had its moment in early-to-mid 2025 and the crowd has partly drifted to LTX-2's native audio or InfiniteTalk for talking-head generation. But for the specific, cost-effective job of re-voicing footage you already have - no API, no key, no driving video - this wrapper still gets you there with just a ComfyUI graph.