Nodes/SDXL Auto Prompter/APNext H3 Music Video (Minimal)
ComfyUI Node

APNext H3 Music Video (Minimal)

The APNext H3 Music Video (Minimal) is four knobs and a prayer

By dagthomas·Created 3 years ago·Updated 12 days ago· 290
APNext H3 Music Video (Minimal)
  • audio
  • llm
  • image_1
  • image_2
  • image_3
  • image_4
  • image_5
  • image_6
  • image_7
  • image_8
  • image_9
  • scenes
  • durations
  • lengths
  • audio_segments
  • scenes_text
  • session_id
  • info
  • image_1
  • image_2
  • image_3
  • image_4
  • image_5
  • image_6
  • image_7
  • image_8
  • image_9
  • clip_starts
  • project_name
lyrics
visual_styleLive-action, cinematic
performance80
pace30
modelsonnet
seed-1
prompt_modeRef2VA (bind reference images)
project_name

Here's the whole pitch: drop in a song, paste the lyrics, pick a cinematic look, and the node writes the entire music video. Concept, performer, staging, shot list - the model invents it all. You get one run with three sliders and a clean conscience.

The "Minimal" is doing real work there. This is a one-box wrapper around the pack's full H3 Music Video Writer, which is a deep node with cast sockets, wardrobe locks, location locks and a masked-audio path. The minimal version calls that writer with opinionated defaults so you don't have to. It's a MiniMax-H3 prompt writer underneath, not a renderer - it outputs the scene prompts and audio slices; a MiniMax H3 video node renders them, and H3 Scenes Join stitches the clips back into one video with the original song. The pack ships h3_music_video.json and h3_music_video_minimal.json showing the whole chain end to end.

How it works

Your song is cut into consecutive pieces, each no longer than H3 renders in one clip (5–15 s), and the cut points are chosen on the music - onsets, energy steps, section changes, and right before a lyric line when the lyrics are timed. Each piece's length is snapped to H3's frame grid, so every rendered clip is exactly as long as its audio slice and the stitched video never drifts. Each piece becomes one scene: it's tagged <Audio 1> and reused 1:1 as that clip's soundtrack, and the scene's imagery is staged from that piece's lyric lines.

The three sliders map onto the full writer's controls. performance (default 80) decides how much the singer is on camera: 0–33 is Narrative (story visuals, nobody sings on camera), 34–66 Mixed, 67–100 Performance, where the singer lip-syncs the actual lyric lines on camera. pace (default 30) sets how fast it cuts - the code maps it straight onto max segment length, from ~15 s per clip at 0 down to ~6 s at 100. wildness (default 45) runs the pack's usual scale: grounded performance video at 0, fully surreal above 40, where the seed decides which weird events creep in.

The inputs that matter

  • audio - the song (load it with Load Audio). This one's required, obviously.
  • lyrics - one line per line. Timestamps make the sync exact: [0:15] line, 0:15 line, or LRC [00:15.20] line. [Chorus] tags are kept; untimed lines are spread evenly. Empty = instrumental video.
  • visual_style - the whole video's look. The curated cinematic styles each fix camera, lenses and colour; Auto lets the model pick one to fit the song.
  • performance, pace, wildness - the three knobs above.

Reference images (image_1..9) are the one thing worth attaching: they fix the performer's face, and they pass straight through to the video node. Without them, the model invents a performer from the lyrics.

The writing happens in your local Claude Code CLI (see the pack's Claude Code nodes - subscription seat, no API key, no model files), or on any local/API model if you connect an APNext H3 LLM Backend to the llm socket. The model dropdown lists the Claude Code aliases (sonnet / opus / haiku / default) plus Codex and anything your local servers were serving when the page loaded.

Outputs and the honest catches

Wire these to the video node: scenes (one prompt per piece), lengths (H3 frame counts - no math node needed), audio_segments (the matching audio slice per piece, into ref_audio_1), and clip_starts if you're doing your own assembly. scenes_text is the whole thing as readable text for previewing.

Where people get caught out: a long song means lots of clips, and each one is a separate H3 render - a 4-minute track is a serious queue, not a quick test. The node is only as good as its lyrics sync: untimed lines get spread evenly, so a track without timestamps loses the lip-sync precision that's the whole fun of Performance mode. And it uses your Claude Code subscription seat, so a heavy session can hit the rolling 5-hour rate limit mid-run. If you outgrow the four knobs - you want a written concept, a specific cast, wardrobe or location locks, or the masked-audio path for structural lip-sync - graduate to the full H3 Music Video Writer. This one is for the 80% case: song in, video out.

Categorycomfyui_dagthomas/H3

Inputs (19)

NameTypeDefaultDescription
audioAUDIOThe song. It is cut into pieces on the music and every piece becomes one clip.
lyricsSTRINGLyrics, one line per line. Timestamps make the sync exact: `[0:15] line` (or LRC `[00:15.20] line`); section tags like [Chorus] are kept; untimed lines are spread evenly. Empty = instrumental video. The imagery of every scene is staged from its lyric lines.
visual_styleCOMBOLive-action, cinematicThe look of the whole video - the curated cinematic looks (35mm, Wes Anderson, neon noir, ...) each fix style, camera, lenses and colour. Auto lets the model pick one to fit the song.
performanceINT800–100How much the singer is on camera. 0-33 = Narrative (story visuals, nobody sings on camera), 34-66 = Mixed (performance and story alternate), 67-100 = Performance (the singer lip-syncs the lyrics on camera).
paceINT300–100How fast the video cuts. 0 = long, slow pieces (up to ~15 s per clip), 100 = quick cuts (pieces down to ~6 s). The song is still cut ON the music inside that range.
modelCOMBOsonnetWho writes the prompt. sonnet / opus / haiku / fable / default are Claude Code aliases (`default` = whatever the CLI is configured for). `codex` is the OpenAI Codex CLI with its configured model (shown when installed; `codex:<model-id>` in an H3 LLM Backend picks a specific one). ollama: / lmstudio: / local: entries are whatever your local servers were serving when the page loaded; pick one to run fully offline. Anything not listed goes in model_override.
seedINT-1-1–18446744073709550000Controls caching. -1 re-runs every queue.
prompt_modeCOMBORef2VA (bind reference images)Which official prompt guide the scenes follow, and whether the WRITING model is shown the reference pictures. Ref2VA (guide_ref_en.md): pictures are bound as <Picture N> and also sent to the writer, so it describes the real face. Needs a vision-capable model. Ref2VA blind: the video model still receives every picture under the same label, but the writer never sees one - it takes who is in each picture from the cast lines and image_notes. Use it with any model WITHOUT vision (a text-only local model, an uncensored fine-tune), and write an `Image 1: ...` note per picture. FL / T2VA (guide_base_en.md): everything from scratch in words. Pictures are ignored entirely and no <Picture N> label is written - they still pass through the image outputs. Auto picks Ref2VA when pictures are connected, FL otherwise.
llmoptAPNEXT_LLMOptional. Connect an APNext H3 LLM Backend node to write with Ollama, LM Studio, another OpenAI-compatible server or an API model instead of Claude Code. Overrides the model dropdown while connected.
image_1optIMAGEReference image 1: <Picture 1> in the prompt. Connect the same image to image_1 on the MiniMax H3 Reference to Video node, or use this node's image_1 output.
image_2optIMAGEReference image 2: <Picture 2> in the prompt. Connect the same image to image_2 on the MiniMax H3 Reference to Video node, or use this node's image_2 output.
image_3optIMAGEReference image 3: <Picture 3> in the prompt. Connect the same image to image_3 on the MiniMax H3 Reference to Video node, or use this node's image_3 output.
image_4optIMAGEReference image 4: <Picture 4> in the prompt. Connect the same image to image_4 on the MiniMax H3 Reference to Video node, or use this node's image_4 output.
image_5optIMAGEReference image 5: <Picture 5> in the prompt. Connect the same image to image_5 on the MiniMax H3 Reference to Video node, or use this node's image_5 output.
image_6optIMAGEReference image 6: <Picture 6> in the prompt. Connect the same image to image_6 on the MiniMax H3 Reference to Video node, or use this node's image_6 output.
image_7optIMAGEReference image 7: <Picture 7> in the prompt. Connect the same image to image_7 on the MiniMax H3 Reference to Video node, or use this node's image_7 output.
image_8optIMAGEReference image 8: <Picture 8> in the prompt. Connect the same image to image_8 on the MiniMax H3 Reference to Video node, or use this node's image_8 output.
image_9optIMAGEReference image 9: <Picture 9> in the prompt. Connect the same image to image_9 on the MiniMax H3 Reference to Video node, or use this node's image_9 output.
project_nameoptSTRINGA tag for this run - auto-filled with a random name like 'NeonDollyFoley-7k3q' when the node is created; type your own to rename the project. Wire the node's `project_name` output into Save Video's `filename_prefix` and every clip of the run lands in its own subfolder (output/video/<name>/), so the output folder shows at a glance which videos belong together. Saved scene bundles carry it too. Empty = a fresh random name each run (stable when `seed` is fixed).

Outputs (18)

NameTypeDescription
scenesSTRING
durationsFLOAT
lengthsINT
audio_segmentsAUDIO
scenes_textSTRING
session_idSTRING
infoSTRING
image_1IMAGE
image_2IMAGE
image_3IMAGE
image_4IMAGE
image_5IMAGE
image_6IMAGE
image_7IMAGE
image_8IMAGE
image_9IMAGE
clip_startsFLOAT
project_nameSTRING