LanPaint AV Encode

MiniMax H3 is the first open-weight model that natively generates video with sound, which means it's also the first one you can sensibly inpaint in both modalities at the same time. LanPaint_AVEncode is the node that makes that a single sampler pass instead of a knot of spaghetti.

Before this node, an AV inpaint workflow looked like: GetVideoComponents → VAEEncode → SetLatentNoiseMask for the video, MiniMaxAudioEncode → SetLatentNoiseMask for the audio, then a Concat node to jam both latents together - five nodes and a prayer that the mask shapes and latent shapes all line up. LanPaint_AVEncode replaces the entire chain with one node. It takes the video, decodes its frames and its audio track, encodes each with its own VAE, and bundles them into a single nested latent with the video mask and audio mask riding along inside as per-stream noise masks. One latent out, one sampler pass, one decode on the other side.

What it does step by step

Under the hood: it pulls the frames and audio out of the VIDEO reference, encodes the frames with your video VAE, encodes the audio with the MiniMax H3 audio VAE (resampling to the VAE's rate if needed, and upmixing mono to stereo - the H3 VAE expects stereo), then wraps both latents in a ComfyUI NestedTensor alongside the masks. The sampler then treats the whole thing as one conditional sampling problem: the video stream and the audio stream each run on their own shifted sigma schedule, but they're processed in a single pass. That's the real trick - you're not generating video and audio as two separate jobs that happen to look good together, you're conditioning both at once.

The inputs that matter

• video - from the VideoMaskEditor (or a LoadVideo). This is where both the frames and the audio track come from.
• vae - your video VAE.
• audio_vae - the MiniMax H3 audio VAE. Yes, it's a separate checkpoint; don't use the video VAE here.
• mask - per-frame video mask [F, H, W], 1 = regenerate, 0 = keep.
• audio_mask - [F] or [F, 1] at video frame rate, marking which moments of the audio to regenerate.

Output is a single latent, wired into the LanPaint sampler. If the audio mask comes in as a [F, 1] column, the node flattens it for you.

Install

Same pack, same routine: ComfyUI Manager → search "LanPaint", or clone into custom_nodes and restart:

cd ComfyUI/custom_nodes
git clone https://github.com/scraed/LanPaint

The pack itself declares no pip dependencies. For the audio encode to resample audio you'll want torchaudio installed (ComfyUI's own installs usually include it); without it, a rate mismatch between your audio and the VAE's 32 kHz is a hard error rather than a silent fix. You'll also need the MiniMax H3 model and its audio VAE, per ComfyUI's MiniMax docs - that download is measured in tens of gigabytes, not the node's fault.

The trap

The video needs a real audio track - if components.audio comes back empty the node raises "the video has no audio track to encode" rather than quietly producing a silent latent. And if you're using the VideoMaskEditor, remember the audio intervals you paint there are what feed audio_mask; forget to paint any and the audio mask is all zeros, meaning "keep the audio," which is fine if that's the plan and surprising if it wasn't.