Halo-Lipsy v2.4.0

Halo-Lipsy is a one-node pack that makes an AI-generated face talk in sync with an audio track, and it exists because the other lip sync nodes die on AMD. If you've got a Strix Halo APU, a ROCm card, or any unified-memory setup where the standard Wav2Lip wrappers crash with library mismatches, this is the one that actually runs. It's basically Wav2Lip - the 2020 classic - rebuilt to live entirely inside the ComfyUI process instead of shelling out to subprocesses and temp files.

Why it's different. The usual Wav2Lip ComfyUI wrappers escape your venv with subprocess.run(), write zero-byte "ghost files" OpenCV can't read, and cast vfloat16 tensors straight into NumPy, which is exactly what unified memory hates. Halo-Lipsy skips all of it: face detection runs on CPU, Wav2Lip inference on GPU (ROCm translates the CUDA calls), and every tensor goes through .float().cpu().numpy(). No subprocesses, no temp files, no venv escapes. That's the whole pitch, and it's a good one for the people it's aimed at.

How it works

Feed it an images tensor (your generated video frames) and an audio input, and it walks through this: build mel-spectrogram chunks from the audio with librosa, detect a face per frame on CPU, and run the Wav2Lip model over batches of cropped 96×96 faces. The clever part is the composite - it only replaces the mouth region, feathered with a gradient mask, color-matched and sharpened back into the original frame. Your subject's eyes, hair, and background stay untouched. If the audio is silent or no face is found, it passes your frames through unchanged rather than erroring, which is a nice touch. It outputs images and audio, ready to feed a VHS Video Combine or any IMAGE consumer.

The inputs that matter

You only have to connect two things: images and audio. Everything else has a sane default. The few worth touching:

• fps - must match your source video or the sync drifts. This is the one beginners skip and then blame the node.
• sync_offset - shift audio by ±10 frames when lips lag or lead. Negative = audio earlier.
• mel_step_multiplier - mouth speed. <1 slower, >1 faster.
• inference_batch (default 64) - lower it if you hit out-of-VRAM.
• force_cpu - everything on CPU, zero VRAM, much slower.

mode is sequential (frames match audio order) or repetitive (loops your frames to fill the audio), and trim_to_audio decides whether the video gets cut to the audio length or runs on with un-synced pass-through frames. The checkpoint input is basically "auto" unless you've dropped other .pth files in the search paths.

Installing it

ComfyUI Manager, search "Halo-Lipsy", install. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/bkpaine1/Halo-Lipsy
cd Halo-Lipsy
pip install -r requirements.txt

Then grab the model - wav2lip_gan.pth from the original Wav2Lip repo - and drop it in Halo-Lipsy/checkpoints/ or ComfyUI/models/wav2lip/. No model file, no lip sync; the node needs it and won't download it for you. Dependencies are light: librosa, opencv, scipy, torchaudio, mediapipe.

Where you'll get burned

Two real gotchas. First, if you install via ComfyUI Manager, its install.py only pulls librosa, opencv, scipy, and tqdm - not mediapipe or torchaudio. The node falls back to OpenCV Haar detection, which is noticeably worse on AI-generated faces, so if faces aren't being found, pip install mediapipe yourself. Second, the best face detector (S3FD) only loads if the face_detection module is importable, which it isn't out of the box. Default MediaPipe is fine for most work.

Also set expectations: Wav2Lip is old, and the output is "mouth moves on beat," not deepfake-level realism. If you want cinematic quality, LatentSync is the current favorite, but it's VRAM-hungry and picky - this is the lightweight option that fits next to a loaded HunyuanVideo. It won't wow you, but on AMD it's the one that finishes without the crash dance.