🎵 Music2Video Pipe Expand

The node that unpacks everything Music2Video just made — and costs you nothing

By lazniak·Created 3 days ago·Updated about 12 hours ago· 1

pipe

pipe
image_prompts_start
image_prompts_reference
reference_subjects
video_prompts_i2va
video_prompts_ref2va
negative_prompts
shot_index
start_times
end_times
durations
transcript
analysis_json
audio_clips
clip_prefixes
final_video_name

The Music2Video node sends everything it wrote - image prompts, both MiniMax H3 prompt forms, subject names, per-shot timings, the transcript, the analysis JSON - down one pipe socket. That's a great design for keeping a graph tidy, but you can't wire a pipe into a CLIPTextEncode node. This node is the adapter: it takes the pipe apart and hands each value back out on its own socket. One input, and it calls nothing, generates nothing, and costs nothing - it's pure data plumbing, in the best tradition of the genre.

The thing that makes it genuinely better than a dumb splitter is that it passes the pipe through. Wire pipe in, take durations out next to your sampler, and the same pipe continues down the chain so another expander can grab the transcript for a text preview. Several of these can sit along one wire, tapping off what each part of the graph needs. That's the whole point of a pipe type in the first place: the twelve string and number sockets the main node used to expose were a wall of noodles, and this collapses them without losing anything.

What you'll actually grab

image_prompts_start - one natural-language cinematic prompt per shot, the exact text the built-in renderer would send. Wire it into a CLIPTextEncode and a sampler to render your own start frames locally.
video_prompts_i2va / video_prompts_ref2va - the MiniMax H3 image-to-video and reference-to-video prompts, already in the exact skeleton the model expects. Feed video_prompts_i2va[i] into MiniMaxH3Easy.prompt with mode = image, seconds = durations[i].
durations, start_times, end_times, shot_index - the timing lists, aligned by position. image_prompts_start[3], video_prompts_i2va[3], durations[3] and audio_clips[3] are all about the same shot.
negative_prompts - per-shot merged negatives, ready for the negative encoder.
transcript - the raw Whisper text of the whole track (single string, not per-shot).
analysis_json - everything the run measured and wrote, as one JSON string: BPM, beats, treatment, per-shot fields, what was rendered and what it cost.

There are more outputs than that (reference-sheet prompts, subject names, per-shot audio clips, clip filename prefixes, the final film name) - but those six are the ones that actually feed other nodes. clip_prefixes and final_video_name are the quiet useful ones: drop them into SaveVideo and your clips and film land in the same run's project folder, in shot order, next to the transcript.

Install

It ships in the same pack as the main node, so there's nothing extra: Manager → search Music2Video → install, or git clone https://github.com/lazniak/comfyui-music2video ComfyUI/custom_nodes/music2video, then restart. Needs ComfyUI ≥ 0.3.48 like the rest of the pack.

Gotchas

Only one real one: an output is empty when that part of the run produced nothing. Instrumental track → transcript is empty and reference_subjects is empty. image_prompts_start is never short (a skipped shot falls back to a locally assembled prompt), but the subject lists legitimately can be. And per the README, if you hit a downstream node that's silently skipped rather than run, it's usually this: an empty list output hands out an execution blocker instead of a crashing empty list, so the rest of the graph keeps going. The log names the socket. Read the log, not the canvas.

CategoryMusic2Video

Inputs (1)

Name	Type	Default	Description
pipe	M2P_PIPE		The 'pipe' output of the Music2Video node. Every field is always present in it, so an output here is empty only when that part of the run produced nothing.

Outputs (16)

Name	Type	Description
pipe	M2P_PIPE	The same pipe, untouched, so the next expander down the chain can take its own values out of it.
image_prompts_start	STRING	One start-frame image prompt per shot, in shot order, written by the LLM's image-prompt stage; a shot the model skipped falls back to a string assembled locally from that shot's opening, action and the art direction, so this list is never short. Always produced, whatever 'image_provider' is set to. Wire it into a CLIPTextEncode node (whose CONDITIONING feeds a sampler) or into a text preview node - it is the exact text the built-in renderer sends when 'image_provider' is not 'pipe-steps'.
image_prompts_reference	STRING	One reference-sheet image prompt per subject (not per shot), index-aligned with 'reference_subjects': neutral background, even lighting, whole subject visible, no narrative action. Empty only when no subjects were defined (the subjects stage returned none, or 'max_subjects' is 0). If the reference-prompt stage itself fails, every entry is filled from a locally assembled fallback instead, so the list still matches 'reference_subjects' in length. These are exactly the prompts 'render_subject_sheets' renders into 'subject_images'.
reference_subjects	STRING	The subject names only - not prompts - one per subject, in the same order and length as 'image_prompts_reference'. A subject can be a character, location, prop, vehicle or style; the name is what binds a subject to its <Picture N> reference in the ref2va prompts. Empty when no subjects were defined (the subjects stage returned none, or 'max_subjects' is 0).
video_prompts_i2va	STRING	One MiniMax H3 image-to-video (I2VA) prompt per shot: a header declaring <Picture 1> fully referenced at 0.00 s, then integrated_multimodal_description, overall_soundscape and non_diegetic_music. It assumes the matching entry of 'images' is supplied as the clip's first frame - the <Picture 1> reference is written even on a prompts-only run where no image exists. Always produced; feed it to an image-to-video endpoint together with that start frame.
video_prompts_ref2va	STRING	One MiniMax H3 reference-to-video (Ref2VA) prompt per shot, in the full block form: subject_definitions, summary, retention_analysis, detailed_description, overall_soundscape, non_diegetic_music. Unlike 'video_prompts_i2va' it carries no first frame. Identity comes from reference images cited as <Picture N>, and those citations appear only once the run has rendered subject sheets: with 'video_provider' set and 'video_prompt_source' = ref2va, the strings are re-rendered with the sheet numbers bound in. The same re-render also adds an <Audio 1> definition and retention line when the shot's audio is being sent. Without either, undefined labels are stripped and the subjects are described in words only. Suits minimax/h3/reference-to-video.
negative_prompts	STRING	One negative prompt per shot: 'negative_prompt_base', the run-wide negative the art-direction stage added, and the shot's own, merged with duplicates dropped. It reaches only the built-in start-frame renders on fal: the fal payload builder adds negative_prompt when the endpoint's schema declares it (or when that schema could not be read at all), and it is the first field dropped if the endpoint then refuses the request. OpenRouter's image API is never sent a negative, no video request carries one, and the subject sheets are rendered with the raw 'negative_prompt_base' text only, not with these merged strings. Encode it with CLIPTextEncode and wire the result into a sampler's negative conditioning when you render images yourself.
shot_index	INT	The shot numbers, 1-based and consecutive (1..N), one per shot. The prompts, times, audio clips and images are all in this order and the same length, so use it as the index or label when you fan the lists out into batch nodes. 'videos' is the exception - failed clips are skipped, so it can be shorter.
start_times	FLOAT	Each shot's start in seconds from the beginning of the input track, rounded to 3 decimals, one per shot. The shots are consecutive with no gaps or overlaps, so the first value is 0.0 and each value equals the previous shot's entry in 'end_times'.
end_times	FLOAT	Each shot's end in seconds from the beginning of the input track, rounded to 3 decimals, one per shot. The last value is the full track duration; every other value is the next shot's start.
durations	FLOAT	Shot length in seconds (end minus start, rounded to 3 decimals), one per shot; the planner keeps these inside the 'min_shot_seconds'-'max_shot_seconds' window wherever the track length allows. The video endpoint is asked for this length rounded to whole seconds - on fal then clamped into whatever range that endpoint declares and snapped to the nearest value of its duration list - and the returned clip is trimmed, or its last frame held, back to the exact figure here during concatenation.
transcript	STRING	A single string: the raw Whisper text of the whole track, with no timings. Empty when 'whisper_skip' is on, when transcription failed, or when the track has no vocal. A .txt is written to ComfyUI/output/music2prompts when 'save_transcript' is on: the whole transcript, then one block per shot with that shot's timings, section, start-frame image prompt and the first line of its i2va video prompt truncated to 200 characters (the i2va prompt is logged even when the clips were rendered from ref2va), and the words sung inside that shot.
analysis_json	STRING	A single JSON string holding the whole run: track duration, the librosa analysis, the full transcript with its language and word count, the LLM's interpretation, art direction and subject bible, the descriptions of any wired-in reference images, per-shot timings, section, lyrics and raw LLM fields, the rendering report (providers, models, clip paths) and the main settings. Always produced. The same text is written to ..._analysis.json in ComfyUI/output/music2prompts when 'save_json' is on.
audio_clips	AUDIO	One AUDIO per shot, cut from the input track at that shot's boundaries and widened on both sides by 'audio_clip_padding', with the original sample rate and channel layout untouched. Always produced, even on a prompts-only run. Send them to PreviewAudio / SaveAudio or a lipsync node. These are the same clips the node also offers on its own 'audio_clips' socket - here so a lipsync graph can take them off the pipe next to the prompts instead of running a second wire across the canvas.
clip_prefixes	STRING	One 'filename_prefix' per shot, ready to drop into a SaveVideo (or SaveImage) node: 'music2prompts/<project>_v003/<filename_prefix>_<stamp>_shot001'. It is the same name this node gives the clips it renders itself, so clips rendered elsewhere in the graph - an LTX or Wan subgraph fed from these prompts - land in the same run's folder, in shot order, next to the transcript and the analysis JSON. Relative to ComfyUI's output folder with forward slashes, which is what the save nodes expect on every platform. Always produced, whatever 'video_provider' is set to.
final_video_name	STRING	The 'filename_prefix' for the finished film: 'music2prompts/<project>_v003/<filename_prefix>_<stamp>_final'. Wire it into a SaveVideo node - or leave the Music2Video Concat node's own 'filename_prefix' empty and it reads this off the pipe by itself, so the film lands in the run's project folder under the run's name instead of a folder of its own. Always produced.