Painter MiniMax To Video

PainterMiniMaxToVideo is the "set everything up in one place" node for MiniMax H3, the open-weight multimodal video model that finally brought native stereo sound and reference-to-video to ComfyUI. If H3 is the model you're driving, this is the node that takes your prompt, your reference images, videos and audio, and your first/last frames, and hands back everything a sampler needs - plus a second conditioning for a hires pass. No API key, no cloud call; it runs the whole thing locally through ComfyUI's native H3 support. It's not a loader, so don't expect a model checkpoint inside it. Think of it as the CLIP Text Encode of H3, rewritten by someone who actually got sick of wiring reference conditioning by hand.

The name is honest about what makes it worth grabbing: it's the PainterMiniMaxRefToVideo2 node with first/last-frame hard anchors bolted on, and the hard part is the anchors.

How it works

H3's conditioning is more than a text embedding. The node builds an empty audio-video latent - video and audio latents packed together as a nested tensor (video at 24 fps, audio latent at 40 Hz) - then encodes every reference you give it into the H3 VAE's latent space and tucks them into the conditioning. Reference videos get downsampled to 2 fps frames so the text encoder can see them; reference audio is resampled to the audio VAE's sample rate and encoded separately.

Your first and last frames become minimax_keyframes pinned at frame 0 and the final frame. That's what "hard anchor" means: the model has to start and end exactly on those frames, even if you never mention them in the prompt. And here's the fiddly part the author had to fix: combining keyframes and reference conditioning was broken in stock ComfyUI. Two runtime patches (no files modified on disk) fix a bug where cond_video_latents got overwritten by the refs branch, and a PackedLayout position bug that slid the keyframes off the time axis. If the patches ever fail to apply you'll see a WARNING in the console and your refs + keyframes may misalign - that's your cue to update ComfyUI.

The inputs that matter

The plumbing inputs are clip, vae, and audio_vae - wire them from whatever H3 loader your workflow uses. Then:

• prompt - the real input. Reference things are tagged inline as <Picture 1> through <Picture 9>, <Video k>, <Audio j>. Your first/last frame tags come after the references (<Picture 10>/<Picture 11> when you have nine ref images), though because they're hard-anchored, their tag numbers only matter as long as they don't clash.
• first_frame / last_frame - optional image inputs, the hard anchors.
• ref_image_files - internal JSON behind the node's upload area. Leave it alone; that's for the frontend.
• ref_videos / ref_video_audios / ref_audios - autogrowing inputs, up to three each. Reference videos need at least 5 frames at 24 fps.
• length - frame count at 24 fps. 124 is roughly 5 seconds; the model's trained range is about 124–362 frames. The 17-step widget is H3's temporal latent layout showing through.
• ref_max_size (default 1536) - references are scaled down, never up, to fit the longest edge.

The two-pass trick

The scale parameter exists for a hires workflow. First pass samples the positive and LATENT outputs at base resolution; you upscale the latent, then sample again using upscaled_positive, whose first/last frames were re-encoded at width × scale. Match scale to your latent upscaler's scale - the pack ships a PainterMinimaxH3LatentUpscaler for exactly this - because a low-res keyframe latent against an upscaled latent is a crash waiting to happen, not a quality problem.

Outputs: positive (CONDITIONING) and upscaled_positive (CONDITIONING) for the two samplers, LATENT (the empty AV latent), plus width, height, length, and prompt INTs/STRING if you need them downstream.

Installing it

ComfyUI Manager, search "PainterNodes" - the pack title is ComfyUI-PainterNodes. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/princepainter/ComfyUI-PainterNodes
cd ComfyUI-PainterNodes
pip install -r requirements.txt

Then restart ComfyUI. The dependency list is mercifully short - soundfile>=0.12.1, numpy>=1.21.0 - because the real requirements are ComfyUI with native H3 support (the node imports comfy.ldm.minimax.model and comfy.nested_tensor) and the H3 checkpoint itself, which is a big boy. This node is the lightweight front door; the weight is in the model.

Where people get burned

The classic mistakes: running a ComfyUI too old for native H3 (you'll see the patch warnings or an import error), feeding a reference video shorter than 5 frames, and setting scale without touching the latent upscaler. And set expectations on hardware - H3 is enormous; people run it on a 3060 but slowly, and offloading is your friend. Start short and low, get one clip out, then push the length.