Pulse Lip-Sync Segment (one character)

You rendered a two-hander. H3's audio came out fine, the sync is almost there. So you run the film through LatentSync to sharpen it, and now both characters are mouthing the same line - the lip-sync model repainted every face it could find to whoever was speaking. That's the failure Pulse Lip-Sync Segment exists to stop.

What it actually does

It hands the lip-sync model one character. Not "the film, one face masked" - it cuts that character's frames and that character's own recording out of the rendered film and gives you those two things as images and audio. The model never sees the other person's face, so it can't repaint it.

The escalation path, not the default. H3's reference audio is timbre and character with no frame-level alignment, and there's no audio-driven latent path in the public API - so lip sync is something the model does, not something you hand it. Pulse Render's audio_mode: lock_source pins the real waveform into the target audio latent, and that's usually plenty. Fast line, profile shot, small face in frame - that's when you reach for a dedicated lip-sync model.

How the cut is chosen

Give it the timeline from Pulse Slate and Pulse Render's frames, and a speaker name. It then:

• Resolves the character. speaker is the @Name the Asset Bin shows for their face, not their audio file's name. Their voices are the ones a PulseVoice binds to them, plus any recording wired to a shot where they're the first speaker.
• Measures the recording, not the window. The level is read off their film-clock track in 10 ms hops. At or above threshold_db is speech; a gap under 0.30 s is bridged as a breath between words, a blip under 0.10 s dropped as a click.
• Adds handles that respect the other actor. Every line is widened by handle_seconds either side - but a handle only reaches into silence. If the other character is audible there it stops, so it can't close their mouth at a cut.
• Cuts picture and sound together. Spans are merged; with region: box the crop is limited to that face. The same seconds come out of the recording, joined, sample rate relabelled by lipsync_fps / 24 (24 → 25 by default) - no resampling, so every mouth shape stays on its frame.

The inputs worth touching

threshold_db (default −40) is the trim dial: room tone reading as speech, raise it; a quiet close-mic'd take falling below the line, lower it. handle_seconds at 0.25 is right for most dialogue - shorter for clipped, overlapping banter, longer if the model chews the first frame of each span.

region plus box_x / box_y / box_w / box_h (0–1 fractions of the frame) is the two-shot switch. Set box and give each character half the frame - the shipped example uses 0, 0, 0.5, 1 and 0.5, 0, 0.5, 1 - so the model only ever sees one face. audio_source picks what it hears: final uses each PulseVoice's clean final_audio take where one exists, drive uses the recording that drove H3.

The outputs: images and audio go into your lip-sync node (LatentSync and friends), segment into Pulse Lip-Sync Paste, report into anything that prints text. Read the report - it's how you find out a character had three lines where you thought they had five.

Install

Same pack, same install as everything else here. ComfyUI Manager, search the pack title - or:

cd ComfyUI/custom_nodes
git clone https://github.com/Addis-Pulse-Studio/comfyui-pulse-studio

Restart ComfyUI. There is nothing to pip install - requirements.txt is a comment block explaining that torch, numpy, Pillow and av are already present in any working ComfyUI, and pinning them would only let the pack fight your host install. You do need ComfyUI 0.30.0 or newer, a build carrying comfy_extras/nodes_minimax_h3.py and comfy/ldm/minimax/, or there's no H3 to direct. The weights are separate downloads from Comfy-Org/MiniMax-H3 into models/*/minimax/: the ref2va and fl2va DiTs at ~20 GB each, the 15 GB text encoder, two VAEs. Linux and macOS users see the shipped graphs' loader widgets load red, because those paths were authored with Windows separators - click each loader, pick the same file, carry on.

example_workflows/PulseSlate_Dialogue_HyperFlow.json ships this whole chain - Segment → LatentSync → Paste, twice - bypassed. Select the group and Ctrl+B to switch it on.

Where it goes wrong

It refuses rather than guessing, and the error text is unusually good. A speaker that matches nothing, or matches an audio asset instead of a face, gets told to use the Asset Bin's @Name. No lip_sync recording bound to them: it says that too. Nothing above threshold_db: it tells you to lower it.

Two real traps. First, the images you wire in must be the film those frames came from - hand it an upscaled or interpolated version and the frame count won't match the timeline, so the spans get clipped and it warns you on the node face. Second, the batch it emits is not a contiguous clip; it's a stack of non-adjacent frames. That's the design, and it's why handle_seconds matters - the handles are the model's run-up. If your lip-sync node pads its output to its own grid, don't panic: Paste maps it back by nearest frame and says so.