MiniMax H3 Combined Image And Reference to Video

MiniMax H3 keyframes and references, finally in one node

By entropicnoise·Created 29 days ago·Updated 14 days ago· 4

clip
vae
audio_vae
first_frame
middle_frame
last_frame
ref_images
ref_videos
ref_video_audios
ref_audios

positive
LATENT

◄prompt—►

◄resolution_sourceuse provided width and height values►

◄width1344►

◄height768►

◄length124►

◄reference_resize_moderesize to start frame total_pixels►

MiniMax H3, the 33B omni-modal video model ComfyUI supports natively, conditions two different ways. You can pin images to specific spots on the output timeline (the "FL2VA" first/last-frame path), or hand it unpinned reference media - identity shots, motion clips, even audio - that steer the result without owning a frame (the "Ref2VA" path). ComfyUI ships a separate node for each. This pack fuses both into one, adds a middle keyframe, and throws in a true single-frame mode. Want "start here, end there, and by the way keep this face and that clip's motion" in one generation? This is the node.

Honesty check before you install: if all you do is simple first-frame image-to-video, the stock nodes are fine and you don't need this. It earns its place when you want keyframes and references in one payload - or when you want H3 to generate stills.

What it actually does

Despite the "Video" in the name, this node never generates anything. No API, no key, no model loading. It's a conditioning-builder: give it a clip, a vae, an audio_vae, a prompt, and whatever media you care about, and it returns a positive CONDITIONING and an empty LATENT ready for sampling.

Under the hood it borrows the exact helpers ComfyUI's own H3 nodes use (comfy_extras.nodes_minimax_h3): encodes your keyframe images and references through the VAE, builds an empty audio/video latent sized to your canvas, tokenizes the prompt with the media attached, and stamps the conditioning with minimax_keyframes and minimax_refs payloads. One payload carries both conditioning styles instead of forcing you to chain nodes.

Keyframe placement is simple. first_frame lands on frame 0, last_frame on the final frame, and the newer middle_frame at frame_count // 2 - for the default 124 frames that's frame 62, ~2.6 seconds into a 24fps clip. Set length to 1 and it collapses: a true single-frame video latent (T=1), every connected guide resolves to frame 0, and you're doing T2I / I2I / Ref2I stills with the same omni model. "To Video" is a bit of a lie - happily so.

The inputs that matter

The required inputs are mostly loaders you wire once. The ones you'll actually fiddle with:

length - frame count, default 124. Leave it for video, set it to 1 for a still.
resolution_source - default "use provided width and height values" uses the width/height widgets (1344×768). Switch to "use width and height of source image first" and the canvas follows your first keyframe's aspect ratio - handy for a tall portrait source you don't want letterboxed.
reference_resize_mode - default "resize to start frame total_pixels" is the old match behavior, and it's the sane default: it keeps a reference image about the same latent area as one output frame, never upscales it. "resize MiniMax H3 2k resolution" (the old max) keeps references huge and will cost you VRAM and time. Change this mode and your results will change - that's expected, not a bug.
prompt - this is where the refs get their names. Numbered media inputs map to <Picture 1>, <Video 1>, <Audio 1>, and you can bind subjects with <Subject 1>. The tooltip walks you through the recommended section structure for full-reference prompts (subject_definitions → summary → retention_analysis → …); following it helps identity stick.

The autogrow sockets handle the media: up to 10 ref_image slots, 4 ref_videos, a matching soundtrack slot per video, and 4 standalone ref_audios. Reference videos need at least five frames (~0.2s) or the node throws.

Wiring it in

Both outputs go forward into the H3 sampling side of your workflow - positive into your MiniMax H3 sampler, the LATENT alongside it - and ComfyUI's H3 decode nodes turn the result into video-plus-audio. First H3 graph? Grab ComfyUI's stock MiniMax H3 example workflow and swap its conditioning nodes for this one.

Installing

ComfyUI Manager is the easy path - search "MiniMax H3 Combined Image And Reference to Video". Otherwise clone it in:

cd ComfyUI/custom_nodes
git clone https://github.com/entropicnoise/MiniMax-H3-Combined-Image-And-Reference-to-Video

Restart ComfyUI and you'll find it under model / conditioning / minimax. No pip dependencies - the only real requirement is ComfyUI v0.33.1 or newer, since the node imports ComfyUI's built-in MiniMax H3 support. You still need the H3 weights (~42.5 GB full precision, plus CLIP/VAE/audio-VAE). And the elephant: H3's community license geofences the local weights out of the US, EU, UK and South Korea - in those regions, running them isn't licensed, even if the hardware copes.

Common issues

VRAM climbs with references - the 2K reference resize mode makes big reference latents. Switch to "resize to start frame total_pixels" and the pressure drops.
PackedLayout errors - update the node. Version 1.6 fixed the interior-keyframe marker its middle-frame compatibility path needs, and older cores behave differently.
Your middle frame dominates everything - that's by design. middle_frame is a hard keyframe pinned mid-timeline, not a suggestion. If you want identity steering without nailing an image to the center of the clip, use ref_images instead. Keyframes and references are different tools, and this node assumes you know which you're grabbing.

Last take: this is a fiddly, opinionated node for a specific job - mixing timeline-pinned guides with loose references in one shot is awkward in stock ComfyUI, and this removes the awkwardness. If that's not your job, the built-ins are lighter. If it is, this is the one you'll reach for.

Categorymodel/conditioning/minimax

Inputs (16)

Name	Type	Default	Description
clip	CLIP		—
vae	VAE		—
audio_vae	VAE		—
prompt	STRING		MiniMax H3 prompt guide. Use reference tags to point the prompt at supplied media, for example <Picture 1>, <Picture 2>, <Video 1>, <Audio 1>, and structured subject labels such as <Subject 1>, <Subject 2>. Keep numbering consistent with the order of the corresponding references. Recommended prompt sections depend on the generation mode: 1. T2VA / I2VA / FL2VA / L2VA integrated_multimodal_description → overall_soundscape → non_diegetic_music Use integrated_multimodal_description for the main visual action, subjects, camera, timing, environment, and any image/video references. Use overall_soundscape for diegetic/environmental sound and dialogue. Use non_diegetic_music for score or background music. 2. Full-reference Ref2VA subject_definitions → summary → retention_analysis → detailed_description → overall_soundscape → non_diegetic_music Use subject_definitions to establish referenced subjects and their identities with tags such as <Subject 1>. Use summary for the overall shot intent. Use retention_analysis to state which reference traits, identity, appearance, motion, voice, or other characteristics should be retained. Use detailed_description for the full scene, action, camera, timing, and reference usage. Finish with overall_soundscape and non_diegetic_music for audio.
resolution_source	COMBO	use provided width and height values	Chooses the generation resolution and, for source-resize modes, how first_frame / middle_frame / last_frame are fitted. • use width and height of source image first — first checks first_frame, then middle_frame, then last_frame. If none are connected, falls back to the first visual reference, then provided width/height. • use width and height of reference image first — first checks ref_images, then ref_videos. If none are connected, falls back to source frames, then provided width/height. • use provided width and height values — original behavior. • resize source (keep aspect ratio, fit mode: total pixels) — use the first connected source guide as the aspect-ratio anchor and resize it so its total pixel area matches the provided width × height area. May upscale or downscale. • resize source (keep aspect ratio, fit mode: shortest edge) — resize the source so its shortest edge matches the shortest provided edge. May upscale or downscale. • resize source (keep aspect ratio, fit mode: longest edge) — resize the source so its longest edge matches the longest provided edge. May upscale or downscale. • resize source (fit mode: stretch) — use the provided width/height canvas and stretch connected source guides to it. For the three aspect-ratio source modes, the first connected source guide (first_frame → middle_frame → last_frame) determines one shared generation canvas; all connected source guides are aspect-preserving cover-fitted into that canvas. Final width and height are always corrected to multiples of 32.
width	INT	134432–16384	—
height	INT	76832–16384	—
length	INT	1241–3600	Frame count. length=1 uses the confirmed single-frame MiniMax H3 path (video latent T=1) for T2I / I2I / Ref2I workflows. At length=1, any connected first_frame, middle_frame, and last_frame guides all resolve to target frame 0 and are passed through independently without deduplication. All other lengths use stock ComfyUI MiniMax H3 latent creation and normal frame-grid snapping.
reference_resize_mode	COMBO	resize to start frame total_pixels	How reference images are resized before MiniMax H3 reference encoding. Aspect ratio is always preserved and references are never upscaled. • resize to frame short edge — downscale until the reference short edge fits the generated frame short edge. Useful for keeping reference detail roughly bounded by the smaller frame dimension. • resize to start frame long edge — downscale until the reference long edge fits the generated/start-frame long edge. Useful for a stricter fit when the reference and target have different aspect ratios. • resize to start frame total_pixels — downscale by total pixel area so the reference uses about the same image-token/latent area as one generated frame. This is the former 'match' mode. • resize MiniMax H3 2k resolution — keep the reference as large as possible, only downscaling when its short edge exceeds MiniMax H3's 2048 px reference limit. This is the former 'max' mode and can be significantly slower.
first_frameopt	IMAGE		Optional opening keyframe, conditioned at target frame 0. At length=1 it shares frame 0 with any connected middle_frame and/or last_frame guides.
middle_frameopt	IMAGE		Optional middle target keyframe. It is conditioned at approximately half the generated clip: exact 0-based index frame_count // 2. For example, 124 output frames place it at frame 62 (~2.58 s at 24 fps). The image is aspect-preserving cover-cropped to the output canvas. Current ComfyUI H3 supports arbitrary target keyframe indices natively; this node also includes a compatibility layout repair for older H3 cores that only accepted endpoints. At length=1 this guide resolves to frame 0 and is not treated as an interior guide.
last_frameopt	IMAGE		Optional ending keyframe, conditioned at target frame frame_count - 1. At length=1 this is frame 0, shared with any other connected target guides.
ref_imagesopt	COMFY_AUTOGROW_V3		—
ref_videosopt	COMFY_AUTOGROW_V3		—
ref_video_audiosopt	COMFY_AUTOGROW_V3		—
ref_audiosopt	COMFY_AUTOGROW_V3		—

Outputs (2)

Name	Type	Description
positive	CONDITIONING	—
LATENT	LATENT	—