MiniMax H3 Omni Media Prompt Bridge

Here's a thing that took the community a while to internalise about multimodal video models: the prompt doesn't just describe the shot, it indexes the references. H3 sees <Picture 1>, <Video 2>, <Audio 1> and you have to talk about them by position. Getting that right by hand while you're also juggling nine reference images and three clips is miserable.

This node hands the job to a local vision-language model. It takes the planner's ordered media bundle - the actual images, videos and audio, in order - plus a rough prompt, and returns a rewritten prompt. One string out, and it's written in the reference-ordering conventions H3 expects.

How it works

The bridge is deliberately thin on the ComfyUI side: instead of exposing a dozen media ports for you to wire, it reads the MINIMAX_H3_OMNI_MEDIA_BUNDLE that the Material Planner emits and calls the Prompt Rewriter Omni backend directly, as a Python module. It imports that project's arrange, its defaults, its node progress hook, and its rewrite entry point; it validates that the reference kinds it got back match what it sent; and it converts the planner's canvas into the rewriter's own resolution and duration vocabulary.

Which means: this node is a client, not the model. It ships no weights. If the rewriter pack isn't installed, the model combo literally reads Install MiniMax-H3-Prompt-Rewriter-ComfyUI first and running it raises exactly that. Follow that project for model, quantization and VRAM requirements - the README is explicit that those belong to it, not to this pack.

Because it's a module call rather than a graph node, there's an obvious failure mode: the bridge talks to a specific interface. If upstream renames a function or changes MAX_REFERENCES, you get a compatibility error rather than a silent wrong answer - the pack checks and warns. Small mercy, but the right one.

Inputs and outputs that matter

Required: media_bundle (from Material Planner), task, prompt, model, quantization, greedy, seed, keep_model_loaded. Optional: options, max_frames, bypass.

• task defaults to REF2AV - the reference-to-audio-video task. The options list is populated from the installed rewriter, so if your install exposes others they'll show up here.
• quantization defaults to nf4. That's the sensible default for a VLM you're loading on a consumer card; nf4 at a small size drop is the usual trade.
• prompt is where your rough intent goes. Leave it as a short description of the shot; the point is the model expand it against the references.
• max_frames (1–64, default 8) caps how many frames get sampled out of each video reference. Raise it if short glitches are being missed; it costs time and VRAM.
• bypass is the one you'll use most while iterating: with it on, the node returns your prompt unchanged. That lets you swap it out of a working graph without re-wiring.
• keep_model_loaded default off frees VRAM between runs; turn it on if you're rewriting a batch of segments and reloading is the slow part.

Output is a single rewritten_prompt (STRING). Wire it into the Plan Encoder's prompt, and wire the plan itself into the encoder too - that's the pair of connections that dodges the dependency cycle.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/Songssx/ComfyUI-MiniMaxH3-TimelineDirector.git
git clone https://github.com/pytraveler/MiniMax-H3-Prompt-Rewriter-ComfyUI.git

Restart ComfyUI and search MiniMax H3. The Timeline Director declares no pip dependencies of its own, but the rewriter pack does - install its requirements into the same environment. I'd restart after each clone so a failure points at the right repo.

Common issues

RuntimeError: Install https://github.com/pytraveler/MiniMax-H3-Prompt-Rewriter-ComfyUI first. You're running the node described in its own error message - the rewriter module isn't importable. Check it's in custom_nodes and that its own requirements are installed.

"The Omni media bundle contains N references, but Prompt Rewriter Omni supports at most M." Your plan has more media than the rewriter can consume. Trim the segment's assignments in the planner, or use per-segment plans so each segment carries only what it needs. Related: the reuse of a per-segment prompt_index is the clean fix, not deleting your images.

Rewrites ignore a video that's clearly there. Raise max_frames - a reference sampled at too few frames reads as a still.

Nothing happens at all. If the model combo is on the "install first" sentinel, the node is a stub. Everything else is downstream of that.