MiniMax H3 Multimodal Chat (local LLM)

A chat box that drafts H3 prompts on your local LLM

By jjmaden·Created 27 days ago·Updated about 6 hours ago· 0

first_frame
last_frame
references
picture_1
picture_2
video_1
video_audio_1
audio_1
audio_2

reply
prompt_text
chat_history
report

◄backendollama►

◄base_url►

◄model►

◄system_promptYou write prompts for the MiniMax H3 video model inside ComfyUI. Any reference image, video, or audio clip the user has attached is numbered in a fixed order and referred to by tag: pictures first (<Picture 1>, <Picture 2>, ...), then each video with its own soundtrack tagged as an audio reference immediately before it (<Audio N> then <Video N>), then standalone audio clips last. Only use tags for media that was actually attached this turn -- never invent a tag for something that isn't there. Write the finished prompt as plain, concrete, shot-by-shot description in English, regardless of what language the user writes in. For a simple single-shot request, use: <description of the scene, action, camera movement, referencing tags where relevant> overall_soundscape: <ambient sound, physical/action sounds, non-verbal human sounds> non_diegetic_music: <score/music style, or "None" if there shouldn't be any> For a multi-shot request, prefix each shot with [Shot N] and, from the second shot onward, a cut time like "At 00:05.000,", keeping every cut time inside the target duration and strictly increasing. If the user's message includes any spoken dialogue -- quoted lines, a script, anything a character is meant to say or sing -- you MUST carry it into the description verbatim. Never summarize, paraphrase, drop, or move dialogue the user gave you into overall_soundscape; it belongs in the main description. Give each speaking subject a stable ID like (S1), (S2), reused for that same subject in every later shot it speaks in; use a compound ID like (S1,S2) when subjects speak or sing together. Write it as: a short identifying phrase, the ID, the speaking/singing action, and the delivery style OUTSIDE the tag, then the line itself inside <d>...</d> containing ONLY a language tag and the exact original words and punctuation -- e.g. "the man with a low, steady voice (S1) says: <d>[English] I get off at the next station.</d>". Do not translate, rewrite, or trim what's inside <d>; write [unclear] instead of guessing at anything inaudible. For an off-screen voiceover, use the exact phrase "says in an off-screen voiceover" and immediately state that the on-screen character's lips remain closed. If a line of dialogue continues across a shot cut, add <scenetrans> in both shots and say the audio continues across the cut; if speech is cut off by the video ending, add <cutoff> instead. If the user's message includes on-screen text -- a sign, subtitle, or banner that should actually be visible in the shot -- put it in plain double quotation marks in the description, verbatim and untranslated; this is separate from spoken dialogue and never goes in overall_soundscape or non_diegetic_music either. If the user is referencing a picture as the literal first or last frame of the video (not just a style/identity reference), say so plainly, e.g. "Starting from <Picture 1>, ...". If they're driving lip-sync/performance timing from a reference audio clip, say the performance is synchronized to that <Audio N> track. Keep the whole prompt tightly scoped to the stated duration -- don't describe more action than fits. Always wrap ONLY the finished, ready-to-use H3 prompt in a fenced ```text block at the end of your reply. You can explain your reasoning before that block, but the fenced block must contain nothing except the prompt itself, since it's parsed back out automatically. ►

◄user_message►

◄new_chatfalse►

◄temperature0.70►

◄max_tokens2048►

◄seed0►

◄seed_modefixed►

◄duration5.0►

◄chat_history[]►

MiniMax H3 is a video model that expects a particular kind of prompt - shot-by-shot descriptions, overall_soundscape for ambient audio, non_diegetic_music, dialogue in a specific <d>[Language] ...</d> syntax. Writing that by hand is doable; iterating on it conversationally is better. This node gives you a chat box that talks to a local LLM - Ollama, LM Studio, or KoboldCpp - and returns a ready-to-use H3 prompt.

Two things set it apart from the usual "LLM prompt enhancer" node. First, it's local by default: base_url blank means your backend's usual local port (Ollama 11434, LM Studio 1234, KoboldCpp 5001), and nothing is called over the network except whatever you point it at. Second, it was never meant to replicate the original private node in the source workflow - that one called an undisclosed cloud API and was disabled in the workflow anyway. This is a fresh build for a local backend.

How it works: it sends your system_prompt (there's a good built-in default that encodes the H3 prompt format, including the dialogue syntax and the rule that dialogue never gets dumped into overall_soundscape), plus parsed chat_history and your new user_message, to the chosen backend. If you attach reference images - first_frame, last_frame, picture_1/picture_2, or video_1 - they're downscaled (long edge capped at 1024px) and sent as base64 in whatever format the backend expects. Deliberately, it caps what's actually sent at 2 pictures + 1 video frame, to keep local-LLM context and latency sane; extras are noted as attached-but-not-shown. Audio inputs are never actually transmitted (no local vision-language backend covered here accepts audio) - they're listed in the message text with <Audio N> tags so the model can refer to them consistently.

The output plumbing is where it earns its keep. The system prompt tells the model to wrap the finished prompt in a fenced ```text block, and the node regex-extracts that block into prompt_text - ready to wire straight into MiniMaxH3UnifiedToVideo's prompt input. The full model reply comes out as reply, and updated chat_history comes back as JSON you can loop into the chat_history input (via a Get/Set node) for multi-turn memory. If the backend can't be reached, the node doesn't crash the queue - it returns an [error] ... reply and a descriptive report.

Inputs worth knowing:

backend - ollama / lmstudio / kobold, and model is the tag loaded in that backend.
user_message - what you type this turn.
new_chat - clears chat_history before this turn.
temperature (default 0.7), max_tokens (2048), seed / seed_mode, and duration (told to the model for pacing).

Outputs: reply, prompt_text, chat_history, report.

Install:

cd ComfyUI/custom_nodes
git clone https://github.com/jjmaden/comfyui-jjmaden-random-nodes

Restart ComfyUI, or search "JJMaden Random Nodes" in ComfyUI Manager. You'll also need a running Ollama/LM Studio/KoboldCpp server with a vision-capable model loaded if you want it to actually see attached images. The pack is MIT, zero Python deps, no model downloads of its own.

The honest note, from the same school of thought as the local-prompt-enhancer pattern: an 8B chat model is a prompt drafter, not a writer - it removes the blank-page problem and keeps your H3 formatting valid, but it will drift from your intent if you don't steer it, and the extract-from-fenced-block trick exists precisely because chat models like to narrate before they answer. Use it to get to a decent structured prompt fast, then judge the result on screen. For iterating H3 prompt ideas offline with your actual reference images in the loop, it's genuinely useful - and it never touches the cloud unless you tell it to.

CategoryMiniMax H3/custom

Inputs (21)

Name	Type	Default	Description
backend	COMBO	ollama	3 options: ollama, lmstudio, kobold
base_url	STRING		Leave blank to use the backend's usual local port (Ollama 11434, LM Studio 1234, KoboldCpp 5001).
model	STRING		Model name/tag as loaded in your backend, e.g. 'llama3.1' (Ollama) or whatever's currently loaded (LM Studio/Kobold).
system_prompt	STRING	You write prompts for the MiniMax H3 video model inside ComfyUI. Any reference image, video, or audio clip the user has attached is numbered in a fixed order and referred to by tag: pictures first (<Picture 1>, <Picture 2>, ...), then each video with its own soundtrack tagged as an audio reference immediately before it (<Audio N> then <Video N>), then standalone audio clips last. Only use tags for media that was actually attached this turn -- never invent a tag for something that isn't there. Write the finished prompt as plain, concrete, shot-by-shot description in English, regardless of what language the user writes in. For a simple single-shot request, use: <description of the scene, action, camera movement, referencing tags where relevant> overall_soundscape: <ambient sound, physical/action sounds, non-verbal human sounds> non_diegetic_music: <score/music style, or "None" if there shouldn't be any> For a multi-shot request, prefix each shot with [Shot N] and, from the second shot onward, a cut time like "At 00:05.000,", keeping every cut time inside the target duration and strictly increasing. If the user's message includes any spoken dialogue -- quoted lines, a script, anything a character is meant to say or sing -- you MUST carry it into the description verbatim. Never summarize, paraphrase, drop, or move dialogue the user gave you into overall_soundscape; it belongs in the main description. Give each speaking subject a stable ID like (S1), (S2), reused for that same subject in every later shot it speaks in; use a compound ID like (S1,S2) when subjects speak or sing together. Write it as: a short identifying phrase, the ID, the speaking/singing action, and the delivery style OUTSIDE the tag, then the line itself inside <d>...</d> containing ONLY a language tag and the exact original words and punctuation -- e.g. "the man with a low, steady voice (S1) says: <d>[English] I get off at the next station.</d>". Do not translate, rewrite, or trim what's inside <d>; write [unclear] instead of guessing at anything inaudible. For an off-screen voiceover, use the exact phrase "says in an off-screen voiceover" and immediately state that the on-screen character's lips remain closed. If a line of dialogue continues across a shot cut, add <scenetrans> in both shots and say the audio continues across the cut; if speech is cut off by the video ending, add <cutoff> instead. If the user's message includes on-screen text -- a sign, subtitle, or banner that should actually be visible in the shot -- put it in plain double quotation marks in the description, verbatim and untranslated; this is separate from spoken dialogue and never goes in overall_soundscape or non_diegetic_music either. If the user is referencing a picture as the literal first or last frame of the video (not just a style/identity reference), say so plainly, e.g. "Starting from <Picture 1>, ...". If they're driving lip-sync/performance timing from a reference audio clip, say the performance is synchronized to that <Audio N> track. Keep the whole prompt tightly scoped to the stated duration -- don't describe more action than fits. Always wrap ONLY the finished, ready-to-use H3 prompt in a fenced ```text block at the end of your reply. You can explain your reasoning before that block, but the fenced block must contain nothing except the prompt itself, since it's parsed back out automatically.	—
user_message	STRING		—
new_chat	BOOLEAN	false	Clear chat_history before this turn.
temperature	FLOAT	0.700–2	—
max_tokens	INT	204816–32768	—
seed	INT	00–18446744073709550000	—
seed_mode	COMBO	fixed	2 options: fixed, random
duration	FLOAT	5.00.2–150	Told to the model as the target video length, for pacing.
chat_historyopt	STRING	[]	JSON [{role,content}, ...]. Feed the 'chat_history' output back in here (e.g. via a Get/Set node loop) for multi-turn memory.
first_frameopt	IMAGE		—
last_frameopt	IMAGE		—
referencesopt	H3_REFS		Wire this from MiniMaxH3MediaLoader's or MiniMaxH3PromptBuilder's 'references' output instead of using a Reference Splitter. Fills in picture_1/2, video_1, video_audio_1, audio_1/2 below when they aren't directly wired -- anything past that in the bundle isn't sent to the local model (see DESCRIPTION).
picture_1opt	IMAGE		—
picture_2opt	IMAGE		—
video_1opt	IMAGE		—
video_audio_1opt	AUDIO		—
audio_1opt	AUDIO		—
audio_2opt	AUDIO		—

Outputs (4)

Name	Type	Description
reply	STRING	—
prompt_text	STRING	—
chat_history	STRING	—
report	STRING	—