APNext H3 Masked Song Latent (song frozen in the audio latent)

If you want H3 to lip-sync - real, structural lip-sync where the mouth hits the words because the model can't do otherwise - the trick is to write the actual song slice into the audio latent and freeze it there, protected from denoising. That's H3MaskedSongLatent, the node that made the pack's masked-audio music videos possible without a third-party pack.

What it is

It's a drop-in replacement for MiniMaxH3SongMaskedAVContext from ComfyUI-H3-Motion-Context-MultiRef - same inputs, same three outputs, same chain-continuation paths - so the pack's Chain Render and Scene Retake pick it first and fall back to the MultiRef node when it's missing. It writes this clip's exact song slice into the H3 audio latent using the audio VAE, then protects the audio rows with a nested noise mask: video generated, audio preserved. Nothing is patched; it's all ComfyUI core (comfy.model_base.MiniMaxH3 turning the mask into per-row timesteps).

The part people get wrong: the encoder needs context

The reason this node exists, rather than just "write the slice in and go", is that the H3 audio VAE's encoder is a same-padded conv stack under a causal attention block. Cut the slice hard at the clip start and the first tokens have a zero-padded past and nothing to attend to - the pack measured token 0 ~57% off, the error taking half a second to decay. So:

• preroll_seconds (default 1.0) - encodes that much song before the clip start and drops it, giving the first tokens a real past. 1 s brings the head to ~1% error; 2 s hits the floor. Set it to 0 to reproduce the old hard-cut behaviour for an A/B.
• lookahead_seconds (default 0.2) - the convs look ahead ~200 ms, so the tail of a hard-cut slice is off too. This encodes real song after the cut (silence once the song has ended) and trims it back.

The node handles the 25 ms tick alignment internally - leave the defaults alone unless you're experimenting.

The inputs that matter

• latent - the target AV latent from MiniMax H3 Reference to Video / Image to Video.
• audio_vae - the MiniMax H3 audio VAE, which encodes the song slice.
• master_audio - the whole song (or the conditioning mix from H3 Voice Over Music). Only the slice this clip covers is written in; the rest is ignored.
• clip_start_seconds - where this clip starts in the song. For a continuing clip this already includes the pinned prefix (Chain Render handles that).
• context_length - visual prefix copied from the previous clip: 5 / 22 / 39 / 56… are H3's valid runs; 0 = independent clip.

Optional but genuinely useful: voice - the vocal stem - switches on the voice gate. H3 has one audio latent and it's a mix, so the voice can't go to the lips and the music to the motion separately - but the freeze can follow the voice in time: wherever the stem is sounding, the audio rows hold at audio_denoise (0 = fully frozen, so the lips get the exact vocal); between phrases they relax to gap_denoise (0.15), letting the model re-touch the music bed where no lip-sync is at stake. gate_hold_seconds (0.2) widens the frozen region around each phrase; a small audio_denoise (0.05–0.15) is the A/B knob for letting the model re-touch the audio while the vocal stays.

For chaining, source_latent (previous sampled latent, copied straight - no decode/re-encode) is preferred; source_frames plus vae is the decoded-frames route.

Outputs

latent goes into the sampler, trim_frames is the count to trim, and clip_audio is the song slice for this clip. That's the whole contract - drop-in compatible with the node it replaces.

Installing

Part of the dagthomas pack - ComfyUI Manager, search "comfyui_dagthomas", install, restart:

cd ComfyUI/custom_nodes
git clone https://github.com/dagthomas/comfyui_dagthomas
cd comfyui_dagthomas
pip install -r requirements.txt

The heavy lifting is external: you need the H3 model weights and the H3 audio VAE for audio_vae. There's no extra Python dependency in the pack for this node. Old builds of the pack changed the working directory on load and broke ComfyUI's Restart button; long since fixed, but it's why stale forum posts about it linger.

One honest note: this is the most technical node in the audio suite, and Chain Render drives it automatically - you rarely touch the internals unless you're A/B-ing the preroll or tuning the voice gate. The defaults are good, and the tooltips tell you what each knob measured.