KeySync LipSync

KeySync LipSync takes an existing video's frames plus an audio track and re-renders the mouth so it says what the audio says. That's video-to-video dubbing - not talking-head-from-a-still. It won't bring a static portrait to life; it rewrites the mouth region of footage that's already moving. If your mental model of "lip sync in ComfyUI" is Wan S2V or InfiniteTalk, reset it: those generate a face to match audio, while KeySync is the older-school paradigm (the Wav2Lip/MuseTalk lineage, but diffusion-based) where you already have footage and want the words changed.

KeySync itself is a serious piece of research - "A Robust Approach for Leakage-free Lip Synchronization in High Resolution," two-stage keyframe-plus-interpolation diffusion from Imperial College (arXiv 2505.00497). The two things it's actually known for, and where Wav2Lip-class tools fall over: temporal consistency across frames, and not leaking the original video's mouth movement into the new render (that's the "leakage-free" in the title). This wrapper is thin - it doesn't ship weights or rewrite any KeySync code, it just drives the official repo's scripts/infer_raw.sh through a subprocess and turns the result back into ComfyUI types.

The inputs that matter

• keysync_model - the handle from KeySync Model Loader. Wire that first; the loader is the "did my install work" gate.
• images - your clip as IMAGE frames. A Load Video node feeds this directly.
• audio - an AUDIO track, from a Load Audio node. You can feed a TTS voice here; the whole point is the mouth matching whatever audio you give it.
• input_fps - the frame rate your frames were decoded at (default 25). The wrapper writes them to a temp video at this rate.
• compute_until - how many seconds of animation to actually compute (default 30, range 1–300). Shorten it for faster tests; it's the "animation length" knob.

That's the whole panel. Everything else is fixed under the hood: video is normalized to 25 fps, audio to 16 kHz mono WAV, and the run goes through a subprocess with streaming logs. No occlusion handling here - that's the Advanced node's job.

What comes out

Four outputs, and the README's suggested wiring is a good template:

Load Video (Path) -> KeySync LipSync.images
Load Audio (Path) -> KeySync LipSync.audio
KeySync LipSync.images/audio -> VideoHelperSuite Video Combine

• images - the re-synced IMAGE frames. Into a Video Combine (set fps from frame_rate) or a VHS output.
• audio - your original audio, trimmed or padded to match the output clip length. Handy because you can pass it straight into the combiner and not fight timing.
• frame_rate - usually 25; feed it to whatever node assembles the video.
• output_video_path - the path to the intermediate final.mp4 in the task folder, if you'd rather save that than re-encode.

Each run gets its own folder under ComfyUI/output/keysync/task_<timestamp>_<id>/ with the prepared inputs, the final.mp4, and keysync_stdout.log / keysync_stderr.log.

Install - the wrapper is the easy half

cd ComfyUI/custom_nodes
git clone https://github.com/magicwang1111/ComfyUI-Keysync-wrapper
cd ComfyUI-Keysync-wrapper
git clone https://github.com/antonibigata/keysync

Then restart ComfyUI and grab the models (the Download Models node or a git lfs clone - ~24 GB). The part everyone underestimates: the upstream KeySync repo needs its own Python environment - conda with Python 3.11, torch==2.4.1 + cu121, the upstream requirements, and ffmpeg. The wrapper deliberately doesn't install any of that into ComfyUI. You build the KeySync env, then start ComfyUI with that env active (or drop a .venv inside the keysync repo - the wrapper prepends it to PATH automatically).

Where it goes wrong

The telltale errors: KeySync repo path does not exist, scripts/infer_raw.sh not found, keyframes_ckpt does not exist, ffmpeg not found, and the one everyone actually hits - KeySync official inference script requires bash. This pack is Linux/WSL2/Git Bash territory; native Windows without bash won't run it. When the subprocess itself fails, the error includes the stdout and stderr log paths - read keysync_stderr.log, that's where the real reason lives.

One honest warning: this is not realtime. It's a diffusion pipeline with ~13 GB of model weights per stage; the upstream HF demo caps clips at 6 seconds of GPU. Budget minutes per short clip, and test with compute_until cranked down before you commit to a full render.