ComfyUI-Keysync-wrapper

ComfyUI KeySync Wrapper

This custom node package wraps the official KeySync inference pipeline for ComfyUI.

It does not include KeySync weights and it does not rewrite KeySync model code. The first version calls the official scripts/infer_raw.sh script through subprocess.

Nodes

• KeySync Download Models
• KeySync Model Loader
• KeySync LipSync
• KeySync LipSync Advanced

Typical workflow:

Load Video (Path) -> KeySync LipSync.images
Load Audio (Path) -> KeySync LipSync.audio
KeySync LipSync.images/audio -> VideoHelperSuite Video Combine

KeySync LipSync returns:

• images: ComfyUI IMAGE frames
• audio: ComfyUI AUDIO
• frame_rate: usually 25
• output_video_path: path to the intermediate final.mp4

Install The Official KeySync Repo

Clone the original KeySync repository into this wrapper folder:

cd D:/ComfyUI/custom_nodes/ComfyUI-Keysync-wrapper
git clone https://github.com/antonibigata/keysync

Default repo path used by the loader:

D:/ComfyUI/custom_nodes/ComfyUI-Keysync-wrapper/keysync

Download Models

Recommended model directory:

D:/ComfyUI/models/keysync

You can use the KeySync Download Models node with download=True.

Manual download is also supported:

git lfs install
git clone https://huggingface.co/toninio19/keysync D:/ComfyUI/models/keysync

The loader expects:

D:/ComfyUI/models/keysync/checkpoints/keyframe_dub.pt
D:/ComfyUI/models/keysync/checkpoints/interpolation_dub.pt

The default UI intentionally hides repo and model directory fields. The wrapper checks these fixed paths:

KeySync repo: D:/ComfyUI/custom_nodes/ComfyUI-Keysync-wrapper/keysync
Model dir:    D:/ComfyUI/models/keysync

Runtime Notes

KeySync has heavy dependencies. This wrapper intentionally does not install the full official KeySync dependency set into ComfyUI.

Recommended runtime:

• WSL2 or Linux
• ffmpeg available on PATH
• bash available on PATH
• official KeySync environment/dependencies installed according to the upstream README

If you run native Windows ComfyUI, install Git Bash or use WSL2. The official inference script depends on bash.

Output Layout

Each run creates an isolated task folder:

D:/ComfyUI/output/keysync/task_YYYYMMDD_HHMMSS_xxxxxxxx/
  data/
    videos/input.mp4
    audios/input.wav
  logs/
    keysync_stdout.log
    keysync_stderr.log
  final.mp4

The official KeySync repo also receives its normal output under:

<keysync_repo_path>/outputs/<task_id>/

Set keep_intermediate_outputs=False to remove the prepared input data and upstream output folder after final.mp4 is copied.

Parameters

The simple KeySync LipSync node exposes only the everyday parameters:

• input_fps
• compute_until

Use KeySync LipSync Advanced only when you need the official occlusion path:

• fix_occlusion: enables KeySync's official SAM2-based occlusion mask path.
• position: point prompt for the occluding object, for example 450,450.
• start_frame: frame index where the point prompt applies.

Internal wrapper behavior is fixed for the simple node:

• video is normalized to 25 fps
• audio is normalized to 16 kHz mono WAV
• subprocess mode is always used
• intermediate task files are kept for debugging
• no subprocess timeout is enforced by default

Advanced KeySync parameters such as resize_size, chunk_size, cond_aug, and decoding_t are not exposed in v1 because the official infer_raw.sh script currently hardcodes them.

Occlusion And SAM

The official KeySync code uses SAM2 for occlusion handling. The position field is a SAM2 point prompt used to track the object that blocks the face, such as a hand or microphone. This is powerful but awkward as a text field, so it is hidden from the simple node.

SAM3-based tracking may be added later as a separate preprocessing node that outputs an occlusion mask or point prompt. It is not wired into v1 because the official KeySync infer_raw.sh path expects its own SAM2 implementation and checkpoint layout.

Common Errors

Missing KeySync repo:

KeySync repo path does not exist

Missing official script:

scripts/infer_raw.sh not found

Missing checkpoints:

keyframes_ckpt does not exist
interpolation_ckpt does not exist

Missing ffmpeg:

ffmpeg not found. Please install ffmpeg and add it to PATH.

Missing bash:

KeySync official inference script requires bash.

When KeySync fails, the error message includes the stdout and stderr log paths.