Swan MiniMax H3 Audio Drive

MiniMax H3 is the rare open-weights video model that generates audio natively - stereo, synced, no post-pass bolted on. That's a huge win when you want the model to invent sound, and useless when you have a specific track that has to drive the clip: a line of dialogue you already recorded, a song you're building a visual for. A default H3 workflow hands you the latent with generated audio and lets it ride. This node instead takes the AV latent H3's conditioning nodes produce, stuffs your source audio into the audio half, and locks it with a zero-denoise mask so the sampler can't touch it. The video renders to your audio, not to whatever the model hallucinated.

It's a port of VRGameDevGirl's VRGDG MiniMax H3 Audio Drive - the README credits her directly - repackaged into swan7-py's small "Swan Bits" collection. New pack, no community footprint yet, but the idea is sound, and it's the piece a real H3 audio-lock workflow is missing.

How it works

H3's latent isn't one tensor, it's a nested tensor - a video half and an audio half bundled together. That's what "AV latent" means, and it's why this needs a node at all: a plain latent edit can't touch the audio side. The node splits the bundle, encodes your source track with the H3 audio VAE (resampling to the VAE's expected rate first - 32 kHz by default - if the file's rate differs), then fits the encoded audio into the audio half's layout: [batch, channels, stereo, time], repeated across the batch, trimmed or zero-padded along time to match the video length. It rebuilds the nested tensor with your audio in place and attaches a noise_mask - ones over the video (denoised normally), zeros over the audio (locked, effectively zero denoise).

Two details show the author thought about this. The resampled-and-encoded audio is only for conditioning; the original_audio output returns your untouched AUDIO object for the final mux, so what you hear isn't degraded by a VAE round-trip. And the zero-denoise mask freezes the audio while the video denoises around it - that's the whole "drive" trick.

Wiring it up

Three inputs, all required:

• av_latent (LATENT) - the joint AV latent from a MiniMax H3 Reference/Image-to-Video conditioning node. Connect a plain video latent and it raises "expected a joint video+audio latent," which is its way of saying you grabbed the wrong output.
• source_audio (AUDIO) - the track to drive with, from the official Load Audio node.
• audio_vae (VAE) - the H3 audio VAE. Not the video VAE. People get this wrong.

Outputs: audio_driven_av_latent (LATENT), which feeds the sampler in place of the original AV latent, and original_audio (AUDIO), which goes to the video save/combine node so the muxed file uses the clean source. The author's own workflow note, straight from the node description: keep the same AUDIO connected to ref_audio_0 and reference it as <Audio 1> in the prompt, so the model sees and locks the same source.

Install and what it needs

ComfyUI Manager (search "ComfyUI_Swan_Bits") or a manual clone:

cd ComfyUI/custom_nodes
git clone https://github.com/swan7-py/ComfyUI_Swan_Bits

Then restart. The pack itself has no requirements file and downloads no weights - the heavy part is the H3 model you're already running (full precision is reported around 42.5 GB) plus its H3-VAE. One flag before you get excited: the MiniMax H3 Community License excludes the US, EU, UK and South Korea from its Applicable Territory, so in those regions running the local weights isn't licensed. That's a model-licensing problem, not a node problem, but it's the first thing people hit, so this is your heads-up.

Common issues

• Wrong VAE. Feed the image/video VAE and the encode either fails or produces an audio-latent layout that doesn't match the template. The node checks the layout explicitly and errors, which at least makes it diagnosable.
• Duration mismatch. A source longer than the video latent gets trimmed to fit; shorter, it's zero-padded. Because the muxed audio is the untouched original, a long track runs past the end of the video. Match audio length to video length up front - the pack's other node, Swan Audio Info, exists precisely to give you that frame count.
• Missing torchaudio. The module imports it at the top for resampling. ComfyUI's standard install bundles it with torch, so in practice it's there - but a stripped-down or unusual environment without it will fail to load the whole pack, not just this node.