MiniMax-H3 Reference Encode

You've got a MiniMax-H3 reference-to-video graph working. Pictures, maybe a clip and a voice, a prompt that names them by number, and then one reference that isn't pixels at all - a RefMod, a look or a voice somebody encoded once and saved to a file. You drag its output into a writer's strip, it gets described nicely, and the graph dead-ends. MiniMaxH3ReferenceToVideo encodes whatever you hand it, and a RefMod is encoded already. There's no socket for a latent.

MiniMax-H3 Reference Encode is the way out. It stands in for Reference Slots and MiniMaxH3ReferenceToVideo, takes the same settings, and is the only thing in the pack that can put a saved latent in front of the DiT.

Why the graph needs this at all

H3 is MiniMax's omni-modal video model - native stereo audio generated with the picture, not bolted on after - and references are the part people actually get excited about. The launch-day argument was that reference-to-video eats a lot of jobs people used to train a LoRA for. That only works if the prompt and the reference blocks agree, which is why this pack's writers hand out two things: rewritten_prompt, the text naming each reference (<Picture 1>, <Audio 2>), and references, a typed bundle in strip order.

The normal route is references → Reference Slots (back out to sockets) → MiniMaxH3ReferenceToVideo, which encodes them. This node collapses that: writer's references and prompt in, positive and LATENT out.

Order is the whole contract

H3's DiT has no index for a reference. A block is matched to its label by where it sits - the encoder's <Picture 1> is the first image block. So everything goes in MiniMaxH3ReferenceToVideo's order: every picture, then every clip with its own sound just ahead of it, then every standalone sound. A RefMod takes the place the writer numbered it in. Get that wrong and the video reuses the wrong thing while the prompt politely describes something else - which is what summary is for.

Ordinary references get prepared with that node's own helpers, not a reimplementation: pictures sized to the generation's pixel area (ref_image_size at match) or kept up to a 2048 px short edge (max), clips cut to length. A RefMod skips all of it - its latent becomes the reference block directly - but the text encoder still has to be shown it: a picture as-is, a clip at two frames a second, a stack every fourth photo.

The inputs you'll actually set

• clip - H3's text encoder, nothing else. Load it with a CLIP loader set to minimax_h3. Anything else is refused by name before a model loads, which is the good kind of failure.
• prompt - the writer's rewritten_prompt, unmodified. This is the text whose labels the blocks are matched against.
• width / height / length - mirror what you pass the generator. Defaults are 1344×768 and 124 frames; length is a frame count at 24 fps on H3's 17k+5 grid.
• references - the writer's output. Leave it empty and you get the prompt encoded alone.
• vae / audio_vae - optional and quietly important. Without vae, pictures and clips reach the text encoder only; without audio_vae, sounds do. The summary says how many were left behind. A RefMod reaches the generator either way.

positive goes to the sampler's positive, LATENT is the empty latent you sample into - no separate Empty Latent node needed - and summary is a plain string: wire it to a preview when a reference isn't landing where the prompt says it is.

Install

Same pack, same steps as everything else. ComfyUI Manager → search the pack title, or:

cd ComfyUI/custom_nodes
git clone https://github.com/pytraveler/MiniMax-H3-Prompt-Rewriter-ComfyUI

python_embeded\python.exe -m pip install -r ComfyUI\custom_nodes\MiniMax-H3-Prompt-Rewriter-ComfyUI\requirements.txt

That pulls the pack's own deps (transformers, peft, accelerate, safetensors, bitsandbytes), none of which this node loads for reference handling. What it does need is a ComfyUI new enough for MiniMax-H3 support (comfy_extras.nodes_minimax_h3) plus the H3 video and audio VAEs.

Where people get burned

If the node isn't in the menu, check the console before blaming the clone. The pack imports each node module in its own try/except and logs 'Reference Encode' needs a newer ComfyUI than this one, so it is not registered when the v3 node API isn't there. Everything else still loads, so the pack looks half-installed.

A RefMod with no pictures of its own - most of the ones shared on Hugging Face - has to be decoded for the text encoder. Connect the H3 video VAE here or on its RefMod node, or the node refuses and names the input.

And watch the token bill. A RefMod is not free at sampling time: a ten-photo stack at 1024×1824 is 18,240 tokens riding through every step, as much as eighteen reference pictures. summary prints the real number before you sample. Lowering strength changes how closely it's followed, not what it costs.