Zura · Speech Lip Sync Model

Three choices, one dropdown, and a load call that either works or tells you exactly which file you're missing. That last part is the whole reason this node is worth its own page.

It's a loader, so it doesn't generate anything. It checks that a local lip-sync engine is actually installed, and hands the next node a ZURA_SPEECH_MODEL handle pointing at it. In the pack's own default Studio graph, this node is wired in as Automatic and you never touch it. You touch it when Automatic picks something you didn't want.

The three options

The model widget is an enum with exactly three entries:

• Automatic (the default) - tries LatentSync 1.6 first, and falls back to MuseTalk 1.5 if LatentSync isn't ready.
• LatentSync 1.6 - ByteDance's audio-conditioned lip-sync model. Sharper mouths, and it will refuse to run without a CUDA GPU. It's also the one people discuss: it's the more current of the two in the community, and the failure modes reported for it are the ones you'd expect - lip twitching on long takes, and mouth movement on silent stretches.
• MuseTalk 1.5 - the older, lighter route. Runs on CPU or a non-CUDA device, because the pack only enforces the CUDA requirement on the LatentSync branch. It works, it's softer, and it needs Whisper too.

That Automatic fallback is deliberate and worth understanding: if you install both, you get LatentSync. If you only installed MuseTalk, Automatic silently gives you MuseTalk rather than throwing. If you want to know which one you actually got, wire the ZURA_SPEECH_MODEL output forward and read the receipt your sampler prints.

The mechanism, briefly: both engines are audio-conditioned neural lip sync, and both are run here on face crops, not whole frames. Something upstream has to detect and crop the face - in Zura that's the native WanAnimatePreprocess detection nodes (PoseAndFaceDetection feeds face crops and boxes into the graph) - and something downstream has to paste the animated mouth back into the original frame. This node just decides which engine does the middle part.

Install

Install the pack first, then the models, because the models don't come with it.

cd ComfyUI/custom_nodes
git clone https://github.com/ZURAVFX/ComfyUI_zura_nodes
pip install -r ComfyUI_zura_nodes/requirements.txt

Then run Setup_Speech_Windows.cmd once. On Linux you're doing the equivalent by hand: it downloads into ComfyUI/models/zura_speech/, and the layout it wants is either

models/zura_speech/LatentSync-1.6/latentsync_unet.pt
models/zura_speech/LatentSync-1.6/whisper/tiny.pt
models/zura_speech/sd-vae-ft-mse/{config.json,diffusion_pytorch_model.safetensors}

for LatentSync, or

models/zura_speech/MuseTalk/musetalkV15/{musetalk.json,unet.pth}
models/zura_speech/whisper-tiny/{config.json,model.safetensors,preprocessor_config.json}
models/zura_speech/sd-vae-ft-mse/...

for MuseTalk. Both routes share the sd-vae-ft-mse VAE, so you only download that once.

The helper also refuses to run without its Python packages - diffusers, transformers, huggingface_hub, safetensors, accelerate and einops for the shared path, plus torchaudio, cv2 and numpy for the engines, plus transformers specifically for MuseTalk. Missing ones are printed as a manual pip command rather than silently installed, which is the right behaviour and mildly annoying the first time.

The LatentSync inference code is vendored into the pack (Apache-2.0, ByteDance, bundled licence text included) - no remote inference, no API key, nothing phones home.

Common issues

"Speech lip sync is not installed. Run Setup_Speech_Windows.cmd. Missing: …" - that's the node doing its job. The list after "Missing:" is exact filenames. Ninety percent of the time it's because you downloaded a differently-named checkpoint; the pack matches filenames literally and it does not want you renaming a variant to fit.

"Sharp lip sync needs a CUDA GPU. Select MuseTalk 1.5 for another device." - the LatentSync branch is CUDA-only, hard-stop. On an AMD or CPU-only box, set model to MuseTalk 1.5 and move on.

Setup helper stops halfway. It checks before it downloads, so an incomplete run usually means a missing Python package, not a bad connection. Install what it printed in the same environment that runs ComfyUI - a system pip and a ComfyUI venv pip are not the same thing, and yes, everyone does this once.

One honest caveat

Lip sync is a deepfake-adjacent tool with a real face-consent problem attached, and both engines are trained on faces. Use it on footage you own or have permission to alter. That's not a licence condition on this node, it's just the thing the field keeps having to say out loud.