Extensions/h3-latent-upscaler
ComfyUI Extension

h3-latent-upscaler

MiniMax H3 latent spatial upscale nodes for ComfyUI two-pass video workflows

By rockerBOO·Created 28 days ago·Updated 28 days ago· 4
rockerBOO/h3-latent-upscaler
Nodes4
On cloudLocal install
Categorymodel/sampling/noise/minimax, model/conditioning/minimax
Stars4
Updated28 days ago
Readme

h3-latent-upscaler

ComfyUI custom nodes for spatially upscaling a MiniMax H3 video latent between two sampling passes (low-res -> upscale -> high-res).

Nodes

MiniMax H3 Latent Upscale (model/latent/minimax) Inputs: samples (LATENT, plain video tensor — use after LTXVSeparateAVLatent), scale_by (FLOAT), upscale_method (nearest-exact / bilinear / area / bicubic / bislerp). Output: LATENT. Upscales H/W by scale_by, snapped up to a multiple of 2 in latent space (the DiT's patch size is (1, 2, 2) and unpadded odd sizes will crash).

MiniMax H3 Conditioning Upscale (model/conditioning/minimax) Inputs: conditioning (CONDITIONING), scale_by, upscale_method — use the same values as the paired Latent Upscale node. Output: CONDITIONING. Rescales minimax_refs / minimax_keyframes visual latents (from MiniMaxH3ReferenceToVideo / MiniMaxH3ImageToVideo) so reference/keyframe tokens stay at the right relative scale and RoPE row layout after the video canvas grows. Safe no-op if neither is present — leave it wired even for plain t2va graphs.

MiniMax H3 Add Noise (resample) (model/sampling/noise/minimax) Inputs: model, noise (NOISE), sigmas, latent_image — same inputs as stock AddNoise. Output: LATENT. Drop-in replacement for stock AddNoise when continuing a MiniMax H3 (or any other CONST/flow-parameterized) latent into a second SamplerCustomAdvanced pass via DisableNoise. See "Wiring" below for why stock AddNoise produces corrupted output here.

MiniMax H3 Shift Sigmas (video -> audio) (model/sampling/sigmas/minimax) Inputs: sigmas (SIGMAS), shift_video (FLOAT, default 12.0), shift_audio (FLOAT, default 3.0). Output: SIGMAS. Remaps a video-schedule sigmas array onto MiniMax H3's audio schedule. Insert between the pass-2 BasicScheduler and the audio branch's MiniMaxH3AddNoise only — see "Wiring" below for why the video and audio branches need different sigmas.

Wiring

LTXVSeparateAVLatent / LTXVConcatAVLatent (split/join the NestedTensor AV latent) are stock ComfyUI core nodes that work correctly on MiniMax H3's AV latent as-is. Stock AddNoise, however, does not, for two independent reasons:

  1. Its implementation calls torch.count_nonzero(...) and torch.nan_to_num(...) directly on the latent tensor (comfy_extras/nodes_custom_sampler.py), and neither is one of the operators NestedTensor overloads, so feeding it the joint AV latent crashes with count_nonzero(): argument 'input' ... must be Tensor, not NestedTensor. Fixed by running noise injection on the separated video and audio tensors individually (each a plain Tensor) and only concatenating them back into a NestedTensor afterward, right before the pass-2 sampler.
  2. Even on a plain tensor, stock AddNoise's output is wrong for a CONST-parameterized flow model like MiniMax H3. SamplerCustomAdvanced + DisableNoise internally calls model_sampling.noise_scaling(sigma, zero_noise, latent_image) again as part of its own setup — for CONST that's (1-sigma)*latent_image, silently scaling stock AddNoise's already-correctly-noised output down a second time. Verified against a real run: decoding stock AddNoise's output directly (before the second SamplerCustomAdvanced even runs) showed pure noise at denoise=0.4, and the final video was a corrupted checkerboard/waffle pattern with zero resemblance to the source. Fixed by MiniMaxH3AddNoise — identical to stock AddNoise plus one extra inverse_noise_scaling call that pre-cancels the sampler's redundant rescale. With it, the same denoise=0.4 run produces a correctly upscaled, coherent result.

There is a third, separate issue specific to the audio branch: comfy/ldm/minimax/model.py (core) documents that the DiT receives a single video-schedule sigma from the sampler and internally derives the audio stream's own timestep via a closed-form shift (shift_video=12.0 / shift_audio=3.0 by default — matches the checkpoint's sampling_settings), then rescales the audio velocity it returns to compensate. That only works if the audio latent handed to the model was actually noised to the audio-schedule sigma, not the video one. Feeding the audio branch's MiniMaxH3AddNoise the same sigmas_pass2 as video (as if audio needed "the same noise level") noises it far more than the model expects at that point in the schedule — e.g. at denoise=0.4 (sigma_v=0.4), the model's own math expects the audio latent noised to sigma_a≈0.14, not 0.4. This reads as audio distortion after pass 2. Fixed by MiniMaxH3ShiftSigmas, inserted between the pass-2 BasicScheduler and the audio branch's MiniMaxH3AddNoise only (the video branch keeps the raw sigmas_pass2).

MiniMaxH3ImageToVideo / MiniMaxH3ReferenceToVideo
  -> conditioning, latent
  -> RandomNoise + BasicGuider + SamplerCustomAdvanced   (pass 1)
  -> denoised latent

  -> LTXVSeparateAVLatent(denoised latent) -> video, audio
  -> MiniMax H3 Latent Upscale(video, scale_by, method) -> video_up

  -> MiniMax H3 Add Noise (resample)(model, RandomNoise2, sigmas_pass2, video_up) -> noised_video
  -> MiniMax H3 Shift Sigmas(sigmas_pass2, shift_video=12.0, shift_audio=3.0) -> sigmas_pass2_audio
  -> MiniMax H3 Add Noise (resample)(model, RandomNoise2, sigmas_pass2_audio, audio) -> noised_audio
  -> LTXVConcatAVLatent(noised_video, noised_audio) -> noised_latent_up

  -> MiniMax H3 Conditioning Upscale(conditioning, scale_by, method) -> conditioning_up
  -> rebuild Guider from conditioning_up

  -> SamplerCustomAdvanced (pass 2, DisableNoise + sigmas_pass2 + guider_up + noised_latent_up)
  -> final denoised latent

Example Workflows

example_workflows/ has the two-pass pipeline above already wired into both MiniMax H3 task workflows:

  • minimax_h3_ref2va_two_pass_upscale.json — ref2va (MiniMaxH3ReferenceToVideo)
  • minimax_h3_fl2va_two_pass_upscale.json — fl2va (MiniMaxH3ImageToVideo, wired inside the "Image to Video (MiniMax H3)" subgraph)

scale_by=1.5, bilinear, and the pass-2 BasicScheduler (simple, 15 steps, denoise=0.4) are starting points, not tuned values — adjust by eye once you see a result.

Tests

Run from inside this directory, using ComfyUI's own venv (needed for comfy_api.latest and friends):

cd custom_nodes/h3-latent-upscaler && uv run --project /path/to/ComfyUI pytest -v

pyproject.toml's [tool.pytest.ini_options] sets --import-mode=importlib (pytest's default "prepend" import mode gets confused walking up through this hyphenated package's __init__.py) and -p no:torchtyping (a broken torchtyping pytest plugin registered in the ComfyUI venv otherwise crashes pytest at startup).