H3 Mod Reference to Video

Why you'd reach for it

MiniMax H3 generates picture and sound in one pass, from a prompt that's less "a cat on a skateboard" and more a structured document - summary, detailed_description with [Shot N] markers, overall_soundscape, non_diegetic_music. It also supports reference conditioning natively. What native ComfyUI gives you is sockets to plug references into and a text box to describe them in.

H3 Mod Reference to Video is the workflow layer over that. It keeps ComfyUI's own H3 node underneath - it literally subclasses the native schema, so clip, vae, audio_vae, width, height, length and ref_image_size behave the way the native node does - and adds two things on top: growing sockets for named references and saved characters, and a real compiler that turns your @tags into reference labels, generates the subject and retention sections, and refusal-checks the whole thing before anything runs.

How it works

The added inputs are characters and named_references - autogrow socket groups, prefixed character_ and reference_, up to 16 and 32. Wire H3 Ref Character Picker and the three reference nodes into them. Then budget_mode (Automatic or Manual), manual_reference_budget (default 20480 tokens), saved_character_conditioning (Vision-language + direct references or Direct references only) and allow_reference_overflow, which is the advanced switch that permits reference counts beyond the documented allowances. The tooltip's own caveat is the right one: successful loading does not establish reliable output quality.

At execution the node builds a plan from the tags in your prompt, compiles the text, and encodes each reference group through the native path to get exact token counts. @alias in the prompt expands to <Subject N>; named references become <Picture N>, <Video N>, <Audio N> with independent counters; @alias-voice resolves to the character's voice; @alice: before a line marks who's speaking, and speaker order is inferred from the first explicit vocal event per character unless you annotate it yourself.

Then the checks, in order. Unknown tags raise. Manually annotated speaker numbers that disagree with the inferred order raise. detailed_description and integrated_multimodal_description together raise - pick one. Using <Subject N> by hand raises, because the compiler owns those. And in Manual mode, if the exact reference cost exceeds manual_reference_budget, the node stops before text encoding and names the number. It does not drop anything to fit.

Outputs: positive (CONDITIONING for your sampler, with the reference payloads attached), latent (an empty audiovisual latent for the requested size and length), Prompt (the final expanded prompt) and assignments (labels, associations, token costs). Wire the strings to Inspect H3 Prompt or Assignments.

What people get wrong first

length is frames at 24fps: 124 is about five seconds and the trained range is roughly 124–362. And if you're using endpoint images, width and height must be positive multiples of 32.

ref_image_size is the cost/speed dial. match scales each reference down, keeping aspect, to the generation's pixel area; max uses the reference pipeline's 2048px short edge for best identity fidelity. Reference tokens ride through every sampling step, so max can be several times slower for a modest gain - try match first.

saved_character_conditioning is the other one worth understanding. Both modes supply the saved visual data to the generator; Vision-language + direct references also decodes the final selection so the vision-language encoder can see it. Direct references only skips that decode - and that's the mode that lets saved profiles work without loading a visual VAE at all, because the stored encodings go straight to conditioning.

Install

ComfyUI Manager → search WorkflowX Configurator, or:

cd ComfyUI/custom_nodes
git clone https://github.com/haroonaslam/WorkflowX-Configurator
cd WorkflowX-Configurator && pip install -r requirements.txt

Restart, hard-refresh the browser. You need ComfyUI with native MiniMax H3 support, the H3 text encoder (the node refuses if the conditioning comes back without minimax_token_tags, which means you grabbed something that isn't the H3 encoder), the H3 visual VAE for regular pictures and video and endpoints, and the H3 audio VAE for regular audio and video soundtracks. Saved voices are already encoded and don't need it.

Where it goes wrong

The licence is the elephant. H3's Community License excludes the US, EU, UK and Korea from its Applicable Territory, outputs included, and there's a separate revenue gate on commercial use. Check that before you build a pipeline on it.

Resource-wise, H3 is a 33B model and the reported weight size is around 42.5 GB with no published consumer floor. If a run dies on memory, the cheap wins are a smaller length, ref_image_size=match, fewer saved video samples, and a smaller saved profile - increasing manual_reference_budget does not resolve a reference-count limit, which is a different check.

Finally, one honest expectation: H3 owns appearance, motion and voice generation. Correct associations get your evidence into the prompt; they don't guarantee identity or exact voice cloning.