H3 V2V Init (nested AV latent)
The node that turns a slowed-down clip into a regen ready to go
- samples
- oracle_samples
- mask
- audio_latent
- LATENT
H3 V2V Init is the middle of the MAINodes de-rope pipeline, and it does a job that doesn't look hard until you try it: it takes a VAE-encoded video latent and wraps it as the nested audio+video latent H3's SamplerCustomAdvanced actually expects. MiniMax-H3 denoises video and audio as one joint latent, so you can't just hand it a video tensor and go. This node builds the container, and then it exposes the dials that decide how much of the original survives the regeneration.
The core job
Input samples is the VAEEncode of your smeared frames (from H3 Time Smear). length defaults to 0, meaning "derive the frame count from the latent" - leave it at 0 and only set it to assert a specific grid. The output LATENT goes straight into H3's SamplerCustomAdvanced with a partial-denoise schedule (that's H3 Inject Schedule's job). The audio starts empty and generates jointly with the video on the truncated schedule - causally synced foley, which is the preferred audio source for regenerated content.
The dials a beginner actually sets
oracle_samples+freeze_threshold- background freeze. Wire the baseline latent in, setfreeze_thresholdabove 0 (0.35 is a sane start), and everything outside the oracle's motion region stays frozen to the smeared init. That's the fix for background agents - birds, crowds, traffic - that speed up through a held span: frozen background is held baseline content, so after exact recovery its timing is exactly the baseline's.freeze_grow(default 2) dilates the mask if effects fly too close to the subject and get clipped.mask+invert_mask+mask_feather- manual freeze, overriding the oracle path. The mask marks the region to regenerate; flipinvert_maskto paint the background/birds to freeze directly. Default is hard latent cells (every ~16 px cell fully frozen or fully live, decode smooths the edge); raisemask_featheronly if a hard seam ever shows.time_varying(off by default) keeps a mask's time axis so the freeze region can move - but a moving boundary can pop, so keep transitions on 17-frame phase.audio_latent+audio_mode- the dialogue savior. By default the audio rows start from zeros, so pass 2 invents a fresh performance at natural rate - and recovery then compresses those lips by the hold factor, so held regions come back rushed. That's the whole dialogue defect. Wireaudio_latentfrom H3 Audio Smear → VAEEncodeAudio and setaudio_modeto "follow the original performance (0.5)" and pass 2 renders a genuinely slowed take. Start there; "pin the original outright (0.0)" and the loose 0.7 exist for when the 0.5 bargain isn't right.audio_prefix_tickslets you freeze just the first n ticks of the seeded audio so a continuation carries the previous segment's sound.
Install
Part of matlowai/ComfyUI-MAINodes:
cd ComfyUI/custom_nodes
git clone https://github.com/matlowai/ComfyUI-MAINodes
Restart, under latent/minimax/motion. No extra Python deps for the nodes; example graphs want ComfyUI-KJNodes, and you need MiniMax-H3 weights plus ComfyUI's H3 support (check the licence's US/EU/UK/Korea carve-outs).
The trap everyone hits eventually: the oracle freeze and the manual mask are mutually exclusive paths, and mask overrides the oracle. If you wire a mask and a freeze_threshold and wonder why the threshold did nothing - that's why. Pick one boundary-authoring tool per graph.
Inputs (14)
| Name | Type | Default | Description |
|---|---|---|---|
| samples | LATENT | video latent from VAEEncode of the smeared frames | |
| lengthopt | INT | 00–3600 | 0 = derive from the latent (recommended); nonzero asserts this exact 17k+5 length |
| oracle_samplesopt | LATENT | baseline latent; enables background freezing | |
| freeze_thresholdopt | FLOAT | 0.000–1 | 0 = off. Above 0: freeze background latent to the smeared init so its timing stays exactly the baseline's (fixes background agents speeding up). The subject mask is the oracle heat unioned over time, so the boundary never moves. 0.35 is a sane start. |
| freeze_growopt | INT | 20–16 | latent-pixels of mask dilation (16 image px each); applies to both mask sources |
| maskopt | MASK | (alpha) manual region to REGENERATE (1) vs freeze to baseline timing (0). Overrides the oracle path. Union over time by default: the boundary never moves (set time_varying to keep the time axis) | |
| mask_featheropt | INT | 00–256 | 0 (default): hard latent cells, every cell fully frozen or fully live; the decode smooths the edge. >0: pixel-space ramp pooled to fractional cells (~16 px quanta) if a hard seam ever shows |
| invert_maskopt | BOOLEAN | false | on: the mask marks the FREEZE region instead (paint the background/birds directly) |
| time_varyingopt | BOOLEAN | false | off (default): the manual mask is unioned over time, static boundary. on: a multi-frame mask keeps its time axis, quantized to the latent token grid ((1,4,4,4,4) frames per 17). A moving boundary CAN pop; put intended transitions on 17-frame phase. Manual mask only, ignored by the oracle path and by 2D / single-frame masks |
| audio_latentopt | LATENT | (alpha) VAEEncodeAudio of H3 Audio Smear's output: the baseline track stretched onto THIS dilated clock. Leave unwired for the original behaviour (audio starts from zeros and pass 2 invents its own performance at natural rate, which is what makes held regions come back rushed after recovery) | |
| audio_modeopt | COMBO | custom (use audio_strength) | plain-language presets for the audio rows; anything but 'custom' overrides audio_strength. All but the first need audio_latent wired (H3 Audio Smear -> VAEEncodeAudio). Start with 'follow the original performance': it is what makes a de-roped clip keep its dialogue |
| audio_strengthopt | FLOAT | 1.000–1 | (alpha) how much of the audio rows pass 2 re-renders. 1.0 (default) = unchanged behaviour. With audio_latent wired, 0.5-0.7 keeps the seeded performance's bulk timing and re-renders detail, the same bargain H3 Inject Schedule makes on the video side; 0.0 pins the track outright. H3 runs ONE joint pass, so the sigma schedule cannot set this per modality - it rides the audio half of the noise mask |
| audio_prefix_ticksopt | INT | 00–36000 | (alpha) freeze the FIRST n audio-latent ticks to the seeded audio_latent content (noise-mask 0 there; every later tick keeps audio_strength). The audio twin of the video prefix freeze: a 39-frame carried handle is 65 ticks exactly. Needs audio_latent wired. 0 = off (scalar behaviour unchanged) |
| audio_prefix_release_ticksopt | INT | 00–400 | (alpha) half-cosine release: the LAST n ticks of the frozen prefix ramp 0 -> audio_strength instead of cutting hard (8 ticks = 0.2 s, the field's tested recipe). If used, assembly must let the continuation OWN the overlap tail, or the trim discards the release. 0 = hard edge |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| LATENT | LATENT | — |