H3 Auto Prompt Generator
The VLM that writes MiniMax H3 prompts you'd never type
- reference_image
- video
- frames
- audio
- first_frame_image
- h3_prompt
- h3_prompt_b
- duration_seconds
- fps
- frame_analysis_json
MiniMax H3 is a genuinely good video model, and its prompt format is a genuinely tall order: six lowercase sections, reference labels, retention markers, shot timestamps in a specific syntax, and a mandatory task-type prefix - all of it judged against MiniMax's published writing guide. Writing that by hand for a character-replacement shot is doable. Writing it for every shot, every run, is a job you want a VLM to do.
H3 Auto Prompt Generator is that VLM. It reads a source video plus an identity reference image, picks keyframes on scene cuts and motion peaks, writes a production-ready REF2VA prompt in the official six-section format, then validates and repairs its own output - checking shot times, <Subject 1> tagging, wardrobe mentions, the summary prefix, retention markers, and a character budget - and retries once with the validator's error list. The "auto" in the name isn't marketing; the node is explicitly built so the model's mistakes get caught and fixed before the prompt reaches the sampler.
What you'll actually set
The required inputs are the personality of the prompt: reference_image (identity + wardrobe, <Picture 1>), subject_name (default "Aria Voss" - this becomes <Subject 1> throughout), subject_wardrobe (named 3+ times in the prompt, per the format), and scene_style (one line for detailed_description). Then vlm_provider picks the brain: anthropic, gemini (the only provider that can hear the audio track), openai, kimi, glm, qwen_api, or local options (qwen_local, minicpm_local, magevl_local, ollama). API keys read from env vars (or the optional api_key input).
Three toggles shape the output more than anything else:
cut_times- paste or wire any Cut Detective output here. The cut list becomes ground truth: keyframes land on real shot starts, and the assembler forces the[Shot N]times onto the measured cuts instead of rescaling proportionally. A shot-count disagreement becomes a retry error, not a silent repair. This is the input that makes the pack's Cut Detective + H3 pairing coherent.music_video- writes the prompt as a music video:non_diegetic_musicbecomes the lead audio section and can no longer beN/A, cuts land on the beat, sung words go inlyrics(written as<d>[Language] ...</d>in the shot where they're heard) and the track inmusic_description. Leavelyricsblank and H3 invents nonsense syllables - the tooltip says it plainly.first_frame_alignment- the I2V hook. On,<Picture 1>becomes the literal opening frame; off (the default), it's an identity-only reference whose background must not leak into the video. This is the one people get backwards: on for image-to-video, off for character replacement.
The format details that make it trustworthy
The README is refreshingly specific about what "official" means here: every rule the assembler enforces traces to MiniMax's own VIDEO_PROMPT_WRITING_GUIDE_ref_en.md and ..._base_en.md, and tests/test_h3_format.py asserts each rule against the quoted text. Two things stand out - the trailing block of "No ..." exclusion sentences is not in the official format (it's off by default behind append_exclusions, because H3 has no negative-prompt field), and non_diegetic_music: N/A genuinely means "there is no non-diegetic music."
Outputs: h3_prompt (and h3_prompt_b in both_ab profile mode for A/B testing), duration_seconds, fps, and frame_analysis_json.
Gotchas
- This is a hosted-API node by default: it needs a provider key, and
geminiis the only backend that can describe what's actually heard in the audio. Themodelfield defaults toautoper provider. prompt_profilematters:officialis strict HF-guide format;upgradedadds battle-tested practices (positive assertions, ~3000-char budget, camera speed words);both_abruns two VLM calls so you can compare.- The validator retries once with its error list - so a run can take two API calls. That's a feature (the format gets repaired), just budget for it.
Install
One of ~69 nodes in TrentNodes. The pack install gets you the node; the VLM providers need their Python clients (anthropic, openai, google-genai are in the pack requirements) and keys:
# ComfyUI Manager: search "Trent Nodes"
# or:
cd ComfyUI/custom_nodes
git clone https://github.com/TrentHunter82/TrentNodes.git
cd TrentNodes && pip install -r requirements.txt
No local model downloads unless you pick a local provider. If you've been hand-writing H3 REF2VA prompts, this is the node that makes the format legible instead of a chore.
Inputs (29)
| Name | Type | Default | Description |
|---|---|---|---|
| reference_image | IMAGE | Identity + wardrobe reference (<Picture 1>) | |
| subject_name | STRING | Aria Voss | Name used for <Subject 1> throughout the prompt |
| subject_wardrobe | STRING | charcoal utility jacket over a slate-gray tee, black cargo pants, scuffed black combat boots | Comma-separated wardrobe items exactly as seen in the reference image. Named 3+ times in the prompt. |
| scene_style | STRING | gritty handheld action thriller, overcast daylight, desaturated teal-and-rust color grade | One-line cinematic style for detailed_description |
| soundscape_type | COMBO | ambient | Drives the overall_soundscape guidance |
| vlm_provider | COMBO | anthropic | Hosted APIs: anthropic, gemini (Google; the only provider that can hear the audio input), openai, kimi (Moonshot Kimi K3), glm (Z.ai GLM vision), qwen_api (DashScope intl). Local: qwen_local/minicpm_local/magevl_local (Microsoft Mage-VL 4B, shared with VidScribe) run on this GPU; ollama needs an ollama server. |
| model | STRING | auto | auto = provider default (anthropic: claude-opus-5, gemini: gemini-3.6-flash, openai: gpt-4o, kimi: kimi-k3, glm: glm-4.6v, qwen_api: qwen3-vl-plus, qwen_local: Qwen/Qwen3-VL-8B-Instruct, minicpm_local: minicpm-v-4.5, magevl_local: microsoft/Mage-VL, ollama: qwen3-vl). Or type an explicit model id. |
| max_frames_to_analyze | INT | 82–16 | Keyframes sent to the VLM (plus the reference) |
| enable_audio_prompt | BOOLEAN | true | Off: forces minimal soundscape, no dialogue, music N/A |
| video_mode | COMBO | keyframes | keyframes: send sampled stills (works everywhere). full_clip: send the whole clip so the model reads real motion, cut timing and camera movement instead of inferring them from stills. Video-capable providers only (gemini, kimi); others warn and fall back to keyframes. |
| listen_to_audio | BOOLEAN | true | Send the clip's audio to the VLM so the soundscape describes what is actually heard instead of what the frames imply. Uses the audio input, or the VIDEO's own track when nothing is connected. Audio-capable providers only (gemini today); ignored elsewhere. |
| music_video | BOOLEAN | false | Write the prompt as a music video. non_diegetic_music becomes the lead audio section instead of 'N/A', overall_soundscape thins out to what is audible under the track, cuts are described as landing on the beat, and performance to camera becomes the action. Put the sung words in lyrics and the track in music_description. Connect the audio input too and the prompt declares the song as <Audio 1> reused as the score, with the vocal attributed to the track instead of a new speaker ID. Overrides enable_audio_prompt when they clash. |
| first_frame_alignment | BOOLEAN | false | I2V first-frame hook. Puts 'For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.' above subject_definitions, and rewrites the prompt around it: <Picture 1> becomes the literal opening frame, so its framing, background, lighting and pose are used instead of excluded. Turn this ON for image-to-video, where the reference IS frame one. Leave it OFF for REF2VA character replacement, where <Picture 1> supplies identity only and its background must not leak in. Set the moment with alignment_time_seconds. |
| prompt_profile | COMBO | official | official: HF-guide format with a trailing 'No...' exclusion block. upgraded: same sections + merged battle-tested practices (positive assertions, ~3000-char budget, camera speed words, cut re-anchoring, resolved ending). both_ab: two separate VLM calls; official -> h3_prompt, upgraded -> h3_prompt_b, for A/B testing. |
| videoopt | VIDEO | Source clip (<Video 1>). Preferred input. | |
| framesopt | IMAGE | Alternative to video: an IMAGE batch (e.g. from VHS Load Video). Set fps to match. | |
| fpsopt | FLOAT | 24.001–120 | Frame rate of the frames input (ignored for video) |
| audioopt | AUDIO | Soundtrack to describe. Overrides the VIDEO's own audio. Needs listen_to_audio on and an audio-capable provider. | |
| api_keyopt | STRING | Blank = read the provider's env var: ANTHROPIC_API_KEY / GEMINI_API_KEY (or GOOGLE_API_KEY) / OPENAI_API_KEY / MOONSHOT_API_KEY (kimi) / ZAI_API_KEY (glm) / DASHSCOPE_API_KEY (qwen_api) | |
| dialogueopt | STRING | Exact spoken words, if any. Blank = no dialogue lines are written. | |
| duration_overrideopt | FLOAT | 0.0000–600 | 0 = use the measured clip duration for shot-time validation |
| cut_timesopt | STRING | Measured shot list, normally from Cut Detective's cut_times, shot_table or cuts_json output. Overrides this node's own cut guess: the VLM is told to write exactly these shots, and the assembler forces the [Shot N] times onto them. Accepts seconds ('0, 2.5, 5.083'), MM:SS.mmm timecodes, the shot table, or JSON. Blank = detect cuts locally as before. | |
| lyricsopt | STRING | Exact sung words, in their original language. Written into detailed_description as <d>[English] ...</d> at the shot where they are heard, never repeated in the audio sections. Blank means the mouth moves to the music with no intelligible lyrics - H3 invents nonsense syllables if you ask for singing without giving it words. Needs music_video on. | |
| music_descriptionopt | STRING | The track, for non_diegetic_music: genre, instrumentation, tempo or BPM, and how it develops. Example: 'downtempo synthwave, ~92 BPM, analog pad and gated drums, filter opens into the chorus at the second cut'. Blank lets the model infer it from the attached audio, or from the visuals if none. Needs music_video on. | |
| alignment_time_secondsopt | FLOAT | 0.000–600 | Where <Picture 1> lands on the target timeline when first_frame_alignment is on. 0.00 is the first frame (I2V). A later time anchors the reference mid-clip - the last-frame / FLV trick - and the hook names whichever shot contains it. Ignored when the toggle is off. |
| seedopt | INT | 00–2147483647 | Passed to providers that support seeding. 0 sends no seed at all. The ceiling is the signed 32-bit limit every provider API types this field as; a larger seed from an older workflow is folded into range, not clamped. |
| music_sourceopt | COMBO | auto | Where the music comes from, in music_video mode. auto: the song is declared as <Audio 1> when the audio input is connected, on the assumption that your H3 graph is fed the same file. That is the common case, not a guarantee - set it outright if you know. generate_score: H3 invents the track. reuse_audio_1: the track is supplied to H3 as <Audio 1>, even with nothing wired here. |
| append_exclusionsopt | BOOLEAN | false | Append a trailing block of 'No ...' sentences after non_diegetic_music. OFF by default because it is not part of the official H3 format - no MiniMax guide mentions exclusions, and no official example writes anything after non_diegetic_music. Turn it on if you want the old behaviour: H3 has no negative-prompt field, so some workflows lean on it. |
| first_frame_imageopt | IMAGE | The injected opening frame, for a hybrid graph that feeds H3 both a character reference AND a first frame. Connect it and it becomes <Picture 2>: the prompt pins THIS image to the timeline and opens Shot 1 on it, while <Picture 1> keeps its identity-only role and no shot ever opens on it. Leave it empty and first_frame_alignment aligns <Picture 1> itself, which is right only when your reference IS the opening frame - never when it is a multi-angle character sheet. Wire the same image your H3 encoder gets in slot 2. |
Outputs (5)
| Name | Type | Description |
|---|---|---|
| h3_prompt | STRING | — |
| h3_prompt_b | STRING | — |
| duration_seconds | FLOAT | — |
| fps | INT | — |
| frame_analysis_json | STRING | — |