LatentSync Enhanced (No-Face Safe + OOM Guard)

LatentSync is ByteDance's lip-sync model - one of those open components that shows up everywhere without the parent company's name on it, right up there with Depth Anything and PuLID. It takes footage of a person talking and an audio track, and rewrites the mouth to match the speech while keeping everything else - the pixels, the motion, the lighting - exactly as it was. That "keep the footage, replace the mouth" trick is the whole point of the model, and it's what separates it from the newer native audio+video generators: LTX-2 does synced speech beautifully but regenerates your clip, while LatentSync is the dubbing tool for footage you already love.

The catch has always been the wrapper. LatentSync's original ComfyUI ports were genuinely fragile: crash on a single frame without a face, crash or OOM on anything longer than a few seconds. This node - LatentSync Enhanced - is a rewrite that fixes exactly those two failure modes, which is why it's the one worth reaching for if you're on a consumer GPU.

What actually happens under the hood

The node takes your frames and audio, writes them to temp files, and runs the real LatentSync 1.6 inference pipeline (the latentsync library ships inside the pack - no other custom nodes needed). Audio gets resampled to 16 kHz and fed through Whisper-tiny to produce per-frame audio features. Video gets ping-pong looped to match the audio length (or trimmed if the audio is shorter). Then the diffusion pass happens in segments of chunk_frames frames, each segment detecting faces and doing the mouth-region inpainting at 512×512, before the synced mouth is pasted back onto the original frame.

The two headline fixes, grounded in the code:

• No-face frames - if a frame has no detectable face, the node warns and passes that frame through unchanged, borrowing the nearest face's alignment data so the diffusion sequence stays intact. If no frame has a face, it returns the original video untouched instead of dying.
• OOM control - because processing is segmented, peak VRAM is bounded by chunk_frames, not clip length. It's the dial you turn when a 30-second clip blows up your card.

The inputs that matter

There are seven required inputs; a beginner really touches four of them:

• images (IMAGE) and audio (AUDIO) - your video frames and soundtrack, in that order. Any sample rate works; it auto-resamples.
• chunk_frames - default 80 (≈3.2 s at 25 fps), which the author rates safe for 24 GB VRAM. Drop to 48 at 16 GB, 32 at 12 GB, 16 at 8 GB. This is your OOM escape hatch.
• lips_expression - guidance scale, default 1.5, range 1.0–3.0. Higher = more exaggerated mouth movement; you'll feel this one out per video.
• inference_steps - default 20. More steps, better quality, slower. 20 is a fine place to start.

Also set video_fps to match your source (23.976, 25, 29.97 - it accepts fractional values), and seed if you want reproducible results.

Outputs are images (the synced frames) and audio (resampled to 16 kHz) - wire the images into a video encoder or preview node.

Installing it

Two paths, same result:

• ComfyUI Manager - search "ComfyUI LatentSync Enhanced" and install.
• Manual - cd ComfyUI/custom_nodes && git clone https://github.com/GuardSkill/ComfyUI-LatentSyncEnhanced then restart ComfyUI.

The real work is the model download: about 5.2 GB total into ComfyUI/models/checkpoints/LatentSync-1.6/ - the 4.8 GB latentsync_unet.pt, a whisper/tiny.pt (72 MB), and the vae/ folder (~320 MB). The README's huggingface-cli download ByteDance/LatentSync-1.6 ... command is the reliable way. Put it in the wrong folder and the node throws a helpful FileNotFoundError listing where it searched - it uses ComfyUI's standard checkpoints path, not a pack-local symlink (another fix over the old wrapper). Dependencies come from the pack's requirements.txt: torch, torchaudio, torchvision, diffusers, accelerate, einops, omegaconf, soundfile, plus ffmpeg on your PATH, which it checks for at run time.

Where people get burned

LatentSync in ComfyUI has a reputation as fiddly, and honest community consensus is that newer options like Wan S2V or InfiniteTalk replaced it for many jobs - but those are different tools for different workflows. On this node specifically, the realistic failure list: OOM (turn down chunk_frames), slow runs on older GPUs (it only uses fp16 on CUDA capability > 7, so 10-series cards run fp32 and crawl), and a model that's not installed where it expects it. Oh, and the mouth can look slightly off if the face is tiny in frame - LatentSync works best on close-ups. For an 8 GB card, treat this as a "maybe" and have a shorter clip ready to test.