DiffusionGemma Audio-Aware H3 Multi-Shot Planner

MiniMax H3's native clips are 4–15 seconds. Your music video is 30, 60, 90. Somebody has to divide the one long validated Ref2VA prompt into multiple H3 generation passes - and if that "somebody" is a naive split at 15 seconds, you get cuts landing mid-shot, timestamps going stale, and continuity wrecked. DiffusionGemma Audio-Aware MultiShot Planner is the node that does this division deliberately: it breaks a validated Ref2VA prompt into one to four H3 generation segments, preferring native [Shot N] boundaries, and carries enough state across each seam that later native timestamps stay exact.

It's the planning heart of the music-video stack. A Project Master Contract upstream locks the approved song hash, excerpt, duration, and the hard lane ceiling; the planner respects that ceiling; downstream, the H3 Relay Reference Gate feeds each later lane the previous lane's tail for continuity, and the Shot Assembler joins the segments back into one master.

How it decides where to cut

The planner wants to cut at native shot boundaries with measured low-density recovery (regions where the camera isn't frenetic), then native-only and mixed native/recovery sequences. If no seam can obey every hard lane-duration limit, it falls back to deterministic duration-balanced seams - and crucially, it records each seam's literal provenance and carries the source shot already active at a non-native seam into the next lane, so a shot that starts in lane 2 and ends in lane 3 keeps its exact native timestamps. Timestamped sub-cues get the same treatment: the latest elapsed cue becomes explicit opening-state history, in-lane cues are rebased, future cues are deferred to the later lane rather than firing early or being dropped.

Inputs that matter

• measured_audio_report_json - the decoded-audio measurements (from the selector) that make the plan audio-aware.
• base_h3_prompt - the validated Ref2VA prompt from the Director.
• performance_mode - default Dance / music sync; Natural/Dance/Lyrics travel into each segment's local contract.
• excerpt_start_seconds / excerpt_duration_seconds - where in the soundtrack the plan lives.
• project_manifest_json - the Project Master manifest; its lane ceiling is authoritative.
• preferred_shot_seconds (12) - a legacy, non-binding hint, auto-limited to the active min/max range. min_shot_seconds (5) and max_shot_seconds (15) are the legacy-named hard bounds for one H3 pass; multiple native [Shot N] blocks may remain inside a single pass.
• identity_picture_count - 1 preserves the legacy single-identity anchor; 2 dual-identity mode keeps Picture 1 as body/wardrobe/composition authority and Picture 2 as multi-panel evidence for the same semantic Subject.
• continuity_relay_mode - Previous lane tail preserves the sequential relay; Off removes the continuity Picture from prompts and conditioning so lanes run independently.
• reference_manifest - the exact host manifest used by the Ref2VA Director.
• timed_lyrics_report_json - optional hash-locked lyric alignment evidence. Missing, stale, weak, or invalid evidence safely falls back to Natural/audio-led sync; written lyrics are never treated as a timing schedule.

Outputs include plan_json, effective_shot_count, and per-shot prompt/start/duration/frames/ready for up to four lanes.

The honest edge cases

Lyrics mode accepts only a validated timed-lyrics report: verified events are clipped into each lane's local clock, fully instrumental lanes are marked non-vocal, and anything weak falls back to Natural rather than blocking the video or inventing singing. Structural errors - invalid clocks, duplicate/gapped/empty native shot blocks, out-of-range timestamps, stale audio locks, malformed identity manifests, impossible min/max limits, more than four required lanes - fail closed.

Installing it

One of the pack's prompt/diffusiongemma/production-planning nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/exportAnything/ComfyUI-DiffusionGemmaPromptBuilder

or ComfyUI Manager → DiffusionGemma Prompt Builder → restart. Pure planning logic over JSON - no model of its own. The DiffusionGemma model (ComfyUI/models/LLM/diffusiongemma-26B-A4B-it-NVFP4) runs upstream in the Director; H3 weights matter when you actually render the lanes.

Common issues

• Seams you didn't author. If native shot boundaries can't honor the hard cap, the planner uses deterministic duration-balanced seams and says so in plan_json - check the boundary-policy revision and carry-in fragments before assuming clean shot-to-shot cuts.
• Lane count confusion. effective_shot_count/generation lanes are not the native [Shot N] count - one lane can hold several native shots. Don't feed lane counts back into a Target Profile's shot_count.
• Relay doing nothing. If continuity is off in the plan, the downstream relay gate returns nothing and later lanes run independent - that's configured behavior, not a bug.