H3 Prompt Planner (GGUF)

MiniMax H3 is the omni-modal video model that's been getting real traction since mid-2026 - open weights, plus a ComfyUI path. And like a lot of frontier video models, its prompts are not freeform. The official spec wants a strict three-section structure: integrated_multimodal_description (shot-by-shot, with camera moves, timestamps, and dialogue tags), overall_soundscape, and non_diegetic_music. Writing that by hand, while keeping a video, reference images, and even an audio clip straight in your head, is miserable. This node is a local planner that does it for you: it's the third node in djdzzzz's ComfyUI-Bernini-PromptEnhancer pack, and it runs a Gemma 4 E4B GGUF (4.2 GB, from unsloth) through llama.cpp so you never touch an API.

What sets it apart from the pack's other two nodes is that it takes audio as well as video and images - multi-material context, not just vision. You drag your footage, references, and a soundtrack into the node's panel, reference them in your prompt as <Video 1>, <Picture 1>, <Audio 1>, and it plans an H3-standard prompt with a shot-by-shot breakdown and integrated soundscape.

How it works

The whole node is driven by a custom JS panel (h3_panel.js) that gives you a drag-and-drop media area. Materials get uploaded into ComfyUI's input directory and tracked in a hidden media_manifest widget. On execution, the node reads that manifest, decodes frames from your videos (uniform or smart frame mode), and passes everything to a worker subprocess that subclasses llama-cpp's MTMDChatHandler so Gemma can consume both images and audio through its mmproj. The nice bit: the worker uses existing llama-cpp C bindings for audio, so you don't need to upgrade anything past a normal llama-cpp-python install.

Output comes back as a planning trace and a final prompt. The structured_plan output is the OBSERVATION → UNDERSTAND → EXECUTE → PRESERVE reasoning; the enhanced_prompt is just the final H3 prompt, ready to paste into the official MiniMax H3 Reference to Video node.

The inputs that matter

• model + mmproj - auto-scanned from ComfyUI/models/clip/. The mmproj is required if you want the model to actually see your images/video - the default <none> is text-only, which defeats the point.
• task_type - t2v, i2v, or h3_multi_ref (the default; "everything reference" mode).
• prompt - multiline, and this is where <Video 1> / <Picture 1> / <Audio 1> placeholders go. Plain English instructions work too.
• frame_mode - uniform spacing or smart (key frames - first/middle/end).
• video_frames / sample_fps / sample_seconds - control how much of each video the planner sees. Set sample_fps above 0 and it computes frames as fps × seconds (16 fps × 5 s = 81 frames) instead of a flat count.
• image_max_side - 384/512/768 are the sensible values; bigger is more detail, more VRAM.
• The usual knobs - temperature, repeat_penalty, seed, n_ctx (8192 default), n_gpu_layers, max_tokens, and a thinking_mode toggle.

Outputs

• enhanced_prompt (STRING) - the three-section H3 prompt (FINAL_PROMPT only).
• structured_plan (STRING) - the OBSERVATION–PRESERVE planning trace.
• media_0 … media_14 - passthroughs of everything you uploaded. The panel renames them to ref_video_0, ref_image_0, ref_audio_0, etc., so they slot straight into the official MiniMax H3 node's reference inputs.

Install

Same pack as the other two nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/djdzzzz/ComfyUI-Bernini-PromptEnhancer.git
cd ComfyUI-Bernini-PromptEnhancer
pip install -r requirements.txt

Then grab two files from unsloth/gemma-4-E4B-it-qat-GGUF: the LLM GGUF (4.2 GB) and mmproj-F32.gguf (1.9 GB), both into ComfyUI/models/clip/, then restart ComfyUI so the dropdowns rescan. The node is also installable via ComfyUI Manager under ComfyUI-Bernini-PromptEnhancer.

Gotchas

Here's the one that'll bite you first: requirements.txt for this pack does not list soundfile or scipy, but the audio path imports both. The first time you feed the node an audio clip you'll get No module named 'soundfile' - just pip install soundfile scipy and move on. Expect ~6 GB VRAM total (model + mmproj + KV cache) so a 6 GB card is genuinely right at the edge. And the reload-every-run design from the pack's other nodes applies here too - each execution reloads the 4.2 GB model, so the first queue after a restart is slow. If a video preview shows black, that's your browser refusing H.265/HEVC, not the node failing - it decodes and samples fine regardless.