Nodes/ComfyUI-MAINodes/H3 V2V Init (nested AV latent)
ComfyUI Node

H3 V2V Init (nested AV latent)

The node that turns a slowed-down clip into a regen ready to go

By matlowai·Created 17 days ago·Updated about 16 hours ago· 112
H3 V2V Init (nested AV latent)
  • samples
  • oracle_samples
  • mask
  • audio_latent
  • LATENT
length0
freeze_threshold0.00
freeze_grow2
mask_feather0
invert_maskfalse
time_varyingfalse
audio_modecustom (use audio_strength)
audio_strength1.00
audio_prefix_ticks0
audio_prefix_release_ticks0

H3 V2V Init is the middle of the MAINodes de-rope pipeline, and it does a job that doesn't look hard until you try it: it takes a VAE-encoded video latent and wraps it as the nested audio+video latent H3's SamplerCustomAdvanced actually expects. MiniMax-H3 denoises video and audio as one joint latent, so you can't just hand it a video tensor and go. This node builds the container, and then it exposes the dials that decide how much of the original survives the regeneration.

The core job

Input samples is the VAEEncode of your smeared frames (from H3 Time Smear). length defaults to 0, meaning "derive the frame count from the latent" - leave it at 0 and only set it to assert a specific grid. The output LATENT goes straight into H3's SamplerCustomAdvanced with a partial-denoise schedule (that's H3 Inject Schedule's job). The audio starts empty and generates jointly with the video on the truncated schedule - causally synced foley, which is the preferred audio source for regenerated content.

The dials a beginner actually sets

  • oracle_samples + freeze_threshold - background freeze. Wire the baseline latent in, set freeze_threshold above 0 (0.35 is a sane start), and everything outside the oracle's motion region stays frozen to the smeared init. That's the fix for background agents - birds, crowds, traffic - that speed up through a held span: frozen background is held baseline content, so after exact recovery its timing is exactly the baseline's. freeze_grow (default 2) dilates the mask if effects fly too close to the subject and get clipped.
  • mask + invert_mask + mask_feather - manual freeze, overriding the oracle path. The mask marks the region to regenerate; flip invert_mask to paint the background/birds to freeze directly. Default is hard latent cells (every ~16 px cell fully frozen or fully live, decode smooths the edge); raise mask_feather only if a hard seam ever shows. time_varying (off by default) keeps a mask's time axis so the freeze region can move - but a moving boundary can pop, so keep transitions on 17-frame phase.
  • audio_latent + audio_mode - the dialogue savior. By default the audio rows start from zeros, so pass 2 invents a fresh performance at natural rate - and recovery then compresses those lips by the hold factor, so held regions come back rushed. That's the whole dialogue defect. Wire audio_latent from H3 Audio Smear → VAEEncodeAudio and set audio_mode to "follow the original performance (0.5)" and pass 2 renders a genuinely slowed take. Start there; "pin the original outright (0.0)" and the loose 0.7 exist for when the 0.5 bargain isn't right. audio_prefix_ticks lets you freeze just the first n ticks of the seeded audio so a continuation carries the previous segment's sound.

Install

Part of matlowai/ComfyUI-MAINodes:

cd ComfyUI/custom_nodes
git clone https://github.com/matlowai/ComfyUI-MAINodes

Restart, under latent/minimax/motion. No extra Python deps for the nodes; example graphs want ComfyUI-KJNodes, and you need MiniMax-H3 weights plus ComfyUI's H3 support (check the licence's US/EU/UK/Korea carve-outs).

The trap everyone hits eventually: the oracle freeze and the manual mask are mutually exclusive paths, and mask overrides the oracle. If you wire a mask and a freeze_threshold and wonder why the threshold did nothing - that's why. Pick one boundary-authoring tool per graph.

Categorylatent/minimax/motion

Inputs (14)

NameTypeDefaultDescription
samplesLATENTvideo latent from VAEEncode of the smeared frames
lengthoptINT00–36000 = derive from the latent (recommended); nonzero asserts this exact 17k+5 length
oracle_samplesoptLATENTbaseline latent; enables background freezing
freeze_thresholdoptFLOAT0.000–10 = off. Above 0: freeze background latent to the smeared init so its timing stays exactly the baseline's (fixes background agents speeding up). The subject mask is the oracle heat unioned over time, so the boundary never moves. 0.35 is a sane start.
freeze_growoptINT20–16latent-pixels of mask dilation (16 image px each); applies to both mask sources
maskoptMASK(alpha) manual region to REGENERATE (1) vs freeze to baseline timing (0). Overrides the oracle path. Union over time by default: the boundary never moves (set time_varying to keep the time axis)
mask_featheroptINT00–2560 (default): hard latent cells, every cell fully frozen or fully live; the decode smooths the edge. >0: pixel-space ramp pooled to fractional cells (~16 px quanta) if a hard seam ever shows
invert_maskoptBOOLEANfalseon: the mask marks the FREEZE region instead (paint the background/birds directly)
time_varyingoptBOOLEANfalseoff (default): the manual mask is unioned over time, static boundary. on: a multi-frame mask keeps its time axis, quantized to the latent token grid ((1,4,4,4,4) frames per 17). A moving boundary CAN pop; put intended transitions on 17-frame phase. Manual mask only, ignored by the oracle path and by 2D / single-frame masks
audio_latentoptLATENT(alpha) VAEEncodeAudio of H3 Audio Smear's output: the baseline track stretched onto THIS dilated clock. Leave unwired for the original behaviour (audio starts from zeros and pass 2 invents its own performance at natural rate, which is what makes held regions come back rushed after recovery)
audio_modeoptCOMBOcustom (use audio_strength)plain-language presets for the audio rows; anything but 'custom' overrides audio_strength. All but the first need audio_latent wired (H3 Audio Smear -> VAEEncodeAudio). Start with 'follow the original performance': it is what makes a de-roped clip keep its dialogue
audio_strengthoptFLOAT1.000–1(alpha) how much of the audio rows pass 2 re-renders. 1.0 (default) = unchanged behaviour. With audio_latent wired, 0.5-0.7 keeps the seeded performance's bulk timing and re-renders detail, the same bargain H3 Inject Schedule makes on the video side; 0.0 pins the track outright. H3 runs ONE joint pass, so the sigma schedule cannot set this per modality - it rides the audio half of the noise mask
audio_prefix_ticksoptINT00–36000(alpha) freeze the FIRST n audio-latent ticks to the seeded audio_latent content (noise-mask 0 there; every later tick keeps audio_strength). The audio twin of the video prefix freeze: a 39-frame carried handle is 65 ticks exactly. Needs audio_latent wired. 0 = off (scalar behaviour unchanged)
audio_prefix_release_ticksoptINT00–400(alpha) half-cosine release: the LAST n ticks of the frozen prefix ramp 0 -> audio_strength instead of cutting hard (8 ticks = 0.2 s, the field's tested recipe). If used, assembly must let the continuation OWN the overlap tail, or the trim discards the release. 0 = hard edge

Outputs (1)

NameTypeDescription
LATENTLATENT