Nodes/comfyui-music2video/🎡 Music2Video Pipe Collapse
ComfyUI Node

🎡 Music2Video Pipe Collapse

Put edited prompts back into a Music2Video pipe

By lazniakΒ·Created 3 days agoΒ·Updated about 12 hours agoΒ· 1
🎡 Music2Video Pipe Collapse
  • pipe
  • audio_clips
  • pipe
β—„image_prompts_startβ€”β–Ί
β—„image_prompts_referenceβ€”β–Ί
β—„reference_subjectsβ€”β–Ί
β—„video_prompts_i2vaβ€”β–Ί
β—„video_prompts_ref2vaβ€”β–Ί
β—„negative_promptsβ€”β–Ί
β—„shot_indexβ€”β–Ί
β—„start_timesβ€”β–Ί
β—„end_timesβ€”β–Ί
β—„durationsβ€”β–Ί
β—„transcriptβ€”β–Ί
β—„analysis_jsonβ€”β–Ί
β—„clip_prefixesβ€”β–Ί
β—„final_video_nameβ€”β–Ί

Pipe Expand takes a pipe apart so a value can be used. This node reads that same node backwards: it takes values back in so a value can be changed. Wire in the pipe you started from, wire in only the sockets you actually edited, and everything you left unwired comes through untouched. The run's timings, transcript and per-shot audio stay on the wire, with your edit riding along.

Why does that matter? Because the LLM that wrote your prompts almost never got everything right on the first pass. The typical workflow is: run Music2Video, expand the pipe, hand-edit video_prompts_i2va because shot 7 has the wrong subject in it, and then what? You can't feed an edited string back into the original node - it already ran. Collapse is the bridge: it merges your edited list back into the same pipe data, so everything downstream of the original pipe sees your version of shot 7 and the original everything else. Same trick works for a translation of the prompts, a filtered set of shots, or values produced by another node entirely.

It also builds a pipe from nothing: with no pipe wired in, the fields you supply become the pipe and everything else comes out empty. That's how you hand a graph that never ran the Music2Video node a pipe of its own - for instance, to feed Music2Video Concat shot durations you defined yourself. Like Expand, it calls nothing and costs nothing.

Inputs and output

The optional inputs are the same fields the pipe carries, and they map one-to-one onto what Expand outputs: image_prompts_start, image_prompts_reference, reference_subjects, video_prompts_i2va, video_prompts_ref2va, negative_prompts, shot_index, start_times, end_times, durations, transcript, analysis_json, audio_clips, clip_prefixes, final_video_name - plus the optional pipe to start from. One output: pipe, "the pipe with whatever you wired in, and the rest as it was."

A couple of things worth knowing. Per-shot lists are position-aligned, so if you replace video_prompts_i2va you should replace it with a list of the same length in the same order - a list of edited prompts that's shorter than the shot count will misalign everything downstream. And audio_clips is an AUDIO list, so you can swap per-shot audio too (say, a regenerated vocal slice), not just text. The node is is_input_list, so it handles the whole lists at once.

Install

Same pack, same story: ComfyUI Manager β†’ search Music2Video β†’ install, restart, or git clone https://github.com/lazniak/comfyui-music2video ComfyUI/custom_nodes/music2video. Needs ComfyUI β‰₯ 0.3.48. No extra dependencies, no model downloads.

Gotchas

The practical trap is the inverse of Expand's: if you wire nothing into pipe, you get a new pipe built from your inputs - not the original run's data. So if your goal is "original plus my one edit," always wire the original pipe in. If you wire the pipe in and leave a socket empty, that field comes through from the pipe untouched, which is the behavior you want 95% of the time. And don't forget the media sockets on the main node don't travel this way - audio_clips is in the pipe, but images and videos stay on their own sockets by design, because a rendered clip belongs in a preview or save node, not wrapped in a pipe.

CategoryMusic2Video

Inputs (16)

NameTypeDefaultDescription
pipeoptM2P_PIPEThe pipe to start from. Every field not wired below is taken from it. Leave it empty to build a pipe from nothing, in which case the fields you do not supply come out empty.
image_prompts_startoptSTRINGReplaces this field on the pipe. One start-frame image prompt per shot, in shot order, written by the LLM's image-prompt stage; a shot the model skipped falls back to a string assembled locally from that shot's opening, action and the art direction, so this list is never short. Always produced, whatever 'image_provider' is set to. Wire it into a CLIPTextEncode node (whose CONDITIONING feeds a sampler) or into a text preview node - it is the exact text the built-in renderer sends when 'image_provider' is not 'pipe-steps'.
image_prompts_referenceoptSTRINGReplaces this field on the pipe. One reference-sheet image prompt per subject (not per shot), index-aligned with 'reference_subjects': neutral background, even lighting, whole subject visible, no narrative action. Empty only when no subjects were defined (the subjects stage returned none, or 'max_subjects' is 0). If the reference-prompt stage itself fails, every entry is filled from a locally assembled fallback instead, so the list still matches 'reference_subjects' in length. These are exactly the prompts 'render_subject_sheets' renders into 'subject_images'.
reference_subjectsoptSTRINGReplaces this field on the pipe. The subject names only - not prompts - one per subject, in the same order and length as 'image_prompts_reference'. A subject can be a character, location, prop, vehicle or style; the name is what binds a subject to its <Picture N> reference in the ref2va prompts. Empty when no subjects were defined (the subjects stage returned none, or 'max_subjects' is 0).
video_prompts_i2vaoptSTRINGReplaces this field on the pipe. One MiniMax H3 image-to-video (I2VA) prompt per shot: a header declaring <Picture 1> fully referenced at 0.00 s, then integrated_multimodal_description, overall_soundscape and non_diegetic_music. It assumes the matching entry of 'images' is supplied as the clip's first frame - the <Picture 1> reference is written even on a prompts-only run where no image exists. Always produced; feed it to an image-to-video endpoint together with that start frame.
video_prompts_ref2vaoptSTRINGReplaces this field on the pipe. One MiniMax H3 reference-to-video (Ref2VA) prompt per shot, in the full block form: subject_definitions, summary, retention_analysis, detailed_description, overall_soundscape, non_diegetic_music. Unlike 'video_prompts_i2va' it carries no first frame. Identity comes from reference images cited as <Picture N>, and those citations appear only once the run has rendered subject sheets: with 'video_provider' set and 'video_prompt_source' = ref2va, the strings are re-rendered with the sheet numbers bound in. The same re-render also adds an <Audio 1> definition and retention line when the shot's audio is being sent. Without either, undefined labels are stripped and the subjects are described in words only. Suits minimax/h3/reference-to-video.
negative_promptsoptSTRINGReplaces this field on the pipe. One negative prompt per shot: 'negative_prompt_base', the run-wide negative the art-direction stage added, and the shot's own, merged with duplicates dropped. It reaches only the built-in start-frame renders on fal: the fal payload builder adds negative_prompt when the endpoint's schema declares it (or when that schema could not be read at all), and it is the first field dropped if the endpoint then refuses the request. OpenRouter's image API is never sent a negative, no video request carries one, and the subject sheets are rendered with the raw 'negative_prompt_base' text only, not with these merged strings. Encode it with CLIPTextEncode and wire the result into a sampler's negative conditioning when you render images yourself.
shot_indexoptINTReplaces this field on the pipe. The shot numbers, 1-based and consecutive (1..N), one per shot. The prompts, times, audio clips and images are all in this order and the same length, so use it as the index or label when you fan the lists out into batch nodes. 'videos' is the exception - failed clips are skipped, so it can be shorter.
start_timesoptFLOATReplaces this field on the pipe. Each shot's start in seconds from the beginning of the input track, rounded to 3 decimals, one per shot. The shots are consecutive with no gaps or overlaps, so the first value is 0.0 and each value equals the previous shot's entry in 'end_times'.
end_timesoptFLOATReplaces this field on the pipe. Each shot's end in seconds from the beginning of the input track, rounded to 3 decimals, one per shot. The last value is the full track duration; every other value is the next shot's start.
durationsoptFLOATReplaces this field on the pipe. Shot length in seconds (end minus start, rounded to 3 decimals), one per shot; the planner keeps these inside the 'min_shot_seconds'-'max_shot_seconds' window wherever the track length allows. The video endpoint is asked for this length rounded to whole seconds - on fal then clamped into whatever range that endpoint declares and snapped to the nearest value of its duration list - and the returned clip is trimmed, or its last frame held, back to the exact figure here during concatenation.
transcriptoptSTRINGReplaces this field on the pipe. A single string: the raw Whisper text of the whole track, with no timings. Empty when 'whisper_skip' is on, when transcription failed, or when the track has no vocal. A .txt is written to ComfyUI/output/music2prompts when 'save_transcript' is on: the whole transcript, then one block per shot with that shot's timings, section, start-frame image prompt and the first line of its i2va video prompt truncated to 200 characters (the i2va prompt is logged even when the clips were rendered from ref2va), and the words sung inside that shot.
analysis_jsonoptSTRINGReplaces this field on the pipe. A single JSON string holding the whole run: track duration, the librosa analysis, the full transcript with its language and word count, the LLM's interpretation, art direction and subject bible, the descriptions of any wired-in reference images, per-shot timings, section, lyrics and raw LLM fields, the rendering report (providers, models, clip paths) and the main settings. Always produced. The same text is written to ..._analysis.json in ComfyUI/output/music2prompts when 'save_json' is on.
audio_clipsoptAUDIOReplaces this field on the pipe. One AUDIO per shot, cut from the input track at that shot's boundaries and widened on both sides by 'audio_clip_padding', with the original sample rate and channel layout untouched. Always produced, even on a prompts-only run. Send them to PreviewAudio / SaveAudio or a lipsync node. These are the same clips the node also offers on its own 'audio_clips' socket - here so a lipsync graph can take them off the pipe next to the prompts instead of running a second wire across the canvas.
clip_prefixesoptSTRINGReplaces this field on the pipe. One 'filename_prefix' per shot, ready to drop into a SaveVideo (or SaveImage) node: 'music2prompts/<project>_v003/<filename_prefix>_<stamp>_shot001'. It is the same name this node gives the clips it renders itself, so clips rendered elsewhere in the graph - an LTX or Wan subgraph fed from these prompts - land in the same run's folder, in shot order, next to the transcript and the analysis JSON. Relative to ComfyUI's output folder with forward slashes, which is what the save nodes expect on every platform. Always produced, whatever 'video_provider' is set to.
final_video_nameoptSTRINGReplaces this field on the pipe. The 'filename_prefix' for the finished film: 'music2prompts/<project>_v003/<filename_prefix>_<stamp>_final'. Wire it into a SaveVideo node - or leave the Music2Video Concat node's own 'filename_prefix' empty and it reads this off the pipe by itself, so the film lands in the run's project folder under the run's name instead of a folder of its own. Always produced.

Outputs (1)

NameTypeDescription
pipeM2P_PIPEThe pipe with whatever you wired in, and the rest as it was.