H3 Audio Drive Auto (align + lock)

The mouth is following a voice you can't hear

MiniMax H3 is one of the few open-weights models that generates picture and sound in a single pass. Its joint AV latent carries a video half shaped [B,24,T,H/16,W/16] and an audio half shaped [B,32,2,T*40] - forty audio latent frames per second. The soundtrack isn't a separate stage you can swap afterwards, because by the time you have frames, the mouth has already performed to whatever audio was inside that latent.

That's the whole reason those "add the dub track afterwards" graphs feel off. The mouth is still following the model's own generated voice while your new voice plays over the top. You cannot fix that in the mux.

H3AudioDriveAuto is the unglamorous node that makes the joint pass behave: it takes your wav, encodes it through the H3 audio VAE, and freezes it into the AV latent with a noise mask where video is 1.0 (denoises) and audio is 0.0 (locked). Your voice drives the mouth, and the model can't re-invent it.

Two arithmetic bugs it exists to fix

H3 only accepts frame counts on a 17k+5 grid at 24 fps: 5, 22, 39, 56, … 124, 141. A tidy 5.0 second request is 120 frames, which is not legal. The encoder doesn't complain - it pads or truncates your track behind your back, and you get a mouth that's vaguely wrong for reasons no error message explains. This node computes ceil(duration * 24) and snaps up to the next legal value, then reports which one it used.

The second one is nastier: in a normal graph, audio longer than the video is cut off silently. The tail of your sentence just disappears while the mouth keeps flapping. Here the cut (or the silence pad) is something you choose with pad_mode - silence, trim (shorten the video to the audio), or reject (hard error, my preference while you're setting a shot up) - and the returned waveform is exactly as long as the video, so the final mux physically cannot drift.

The inputs that matter

You need audio (your voice) and audio_vae (the H3 audio VAE, minimax_h3_audio_vae_fp32.safetensors - the fp16 video VAE is a different file and won't do). Then pick a mode:

• lock_source - the default and the one you want. Audio frozen, drives the video, never regenerated.
• remix_source - partial denoise via remix_strength (default 0.35). Keeps the timing, lets the model re-voice it. Strange effect, occasionally useful.
• native - audio half left free, model invents its own track. This is "what H3 does without you".

av_latent is the optional one that changes how you wire the graph: feed it the LATENT from MiniMaxH3ImageToVideo / MiniMaxH3ReferenceToVideo and the audio gets aligned to that latent. Leave it empty and the node builds a clean empty AV latent from width/height plus your audio duration.

audio_offset_frames is the calibration knob, and it's the one that will surprise you. H3 tends to play the mouth roughly ten frames ahead of the track, consistently and linearly. It only shifts the audio you get back - length preserved, conditioning untouched - so -10 lands the lips and the sound on the same frame without changing the performance. Use the number H3LipSyncScore reports for the shot rather than trusting -10 forever.

Outputs: av_latent goes to your sampler's latent input, audio_aligned goes to the mux (VHS_VideoCombine.audio or CreateVideo.audio) - not your raw LoadAudio, which is the whole point - length is the grid-legal frame count, and report is plain text telling you what it trimmed, padded and planned.

One tip from the author that I'd underline: keep the same clip on ref_audio_0 and name it <Audio 1> in the prompt. Otherwise H3 is liable to treat your dialogue as ambience.

Install

ComfyUI Manager → search the pack title H3 Audio Drive Auto → install → restart. Or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/lebao-arch/comfyui-h3-audio-drive-auto
# restart ComfyUI

Nothing heavy gets pulled in - the pack declares no dependencies and needs ComfyUI 0.3.70+ on Python 3.10+. Two practical notes: grab every one of the H3 weights first (diffusion model, the qwen3vl_32b_minimax_h3* text encoder, minimax_h3_video_vae_fp16, minimax_h3_audio_vae_fp32), and remember the node isn't the licensed thing - the H3 weights are, under a community licence that excludes the EU, UK, Korea and the US. Running the node is fine; running the model in those regions is the question you should answer before you build a pipeline on it.

Where it goes wrong

The pack doesn't appear in the menu at all. The Wav2Lip module in this pack imports soundfile at load time, and soundfile isn't part of ComfyUI's core requirements. If the whole pack is missing, pip install soundfile into your ComfyUI env and restart.

ImportError on torchaudio. The resample path imports torchaudio when your clip's sample rate doesn't match the audio VAE's. Most LoadAudio output is 44.1 kHz, so you will hit that line.

Score still 0.00. You're almost certainly muxing the raw clip instead of audio_aligned, or the audio never made it into the latent. Read report - it prints the frame grid, the mask value and the alignment action, which tells you which of those happened.

And the honest limit: lock_source makes the model perform your voice, not phoneme-match it. H3 is not a lip-sync model and doesn't claim to be. If you need the lips on the actual consonants, that's what the pack's Wav2Lip node is for.