DiffSynth: MiniMax H3 Inference

H3 is a 33B omni-modal model: one context holding text, image, video and audio, outputting a clip with native stereo sound. The appeal is obvious (you get picture and sound from one prompt), and so is the catch (33B, and a licence that geofences the US, EU, UK and Korea out of the local-weights grant). Here's what the node actually does with it.

The required inputs

pipe, prompt, seed (default 42), num_inference_steps (default 50), height (768), width (1344). The template's settings are the full-fat path. The featured NF4 template runs 4 steps with the same node, because it stacks a 4-step LoRA - and that pairing is the single most useful thing to copy from it.

negative_prompt defaults to a single space, and at cfg_scale 1 (the default) it is inert. Don't spend time writing one until you've deliberately raised CFG.

Conditioning: keyframes and references

• keyframes (IMAGE) with keyframe_indices - the string of frame indices where those images should land. This is how you pin an opening frame, a closing frame, or both; it's the first-last-frame path the FL2VA variant is named for.
• references (a wildcard socket) with ref_image_short_edge (2048), ref_video_short_edge (768) and ref_video_max_pixels (1032192) - the Ref2VA path. Hand it reference images or video and it conditions on them; the short-edge and pixel caps are there because reference material at full resolution will eat your VRAM before the sampler even starts.
• control_video (IMAGE) with control_scale (1.0) - video-to-video conditioning. H3's V2V motion transfer was one of the things that made it stand out, and this is the pair of inputs for it.
• text_embedding - a socket for a precomputed text embedding, so a repeat run can skip re-encoding the prompt. Only wire it if something upstream produces one; otherwise it stays empty.

Retake

retake_video with frame_regions_to_retake, and retake_audio with retake_audio_sample_rate (32000) and seconds_regions_to_retake - the same idea as LTX-2's retake: hand back the clip and specify a window to redo, in frames for the picture and seconds for the sound. When the audio in one corner of a clip has gone wrong, this is cheaper than a full re-roll.

The sampler dials

flow_shift (12) and audio_flow_shift (3) are the flow-matching schedule shifts for the video and audio towers respectively. The pack defaults are tuned for the distilled 4-step setting, not for a 50-step full-precision run; if you're running 50 steps and the motion looks off, these are the knobs that explain it, not your prompt. cfg_scale (1) is the guidance-distilled default - raising it doubles the per-step cost and overcooks fast.

tiled (on), tile_size (256), tile_overlap (64) for decoding on a card that can't hold the whole latent. use_gradient_checkpointing and use_gradient_checkpointing_offload trade speed for memory - the offload variant is the one that actually rescues an OOM, at the cost of a slower run.

Outputs

Two sockets: video (an IMAGE batch) and audio (an AUDIO). They're generated together and belong together; splitting them downstream is fine, expecting a silent clip if you don't want sound is not - that's what audio_flow_shift and the audio VAE are running for.

Install

ComfyUI Manager → DiffSynth-ComfyUI, or:

pip install git+https://github.com/modelscope/DiffSynth-Studio.git
cd ComfyUI/custom_nodes && git clone https://github.com/modelscope/DiffSynth-ComfyUI.git

Where people get burned

Starting with the full-precision 50-step graph on a consumer card. Use the NF4 template, 4 steps, and learn the node before you go big.

Forgetting everything is one context. References, keyframes and control video all feed the same conditioning; wiring three of them and hoping it takes the best one is not how it works.

50 steps with flow_shift 12. Those aren't a matched pair. Copy the template's combination rather than mixing defaults across the two.

Assuming local weights are licensed everywhere they're downloadable. They aren't, and it's the outputs that are named too.