FL Audio Music Video Sequencer

If you've ever tried to cut an AI music video by hand and had the shots drift out of sync a few bars in, this node is the fix. It's the planning brain in Fill-Nodes' audio-reactive toolchain: give it a track and where the beats fall, and it hands back a shot list - how many shots, how long each one runs, exactly which frames and samples each one spans. It doesn't render anything. It figures out the structure so the downstream nodes can generate clips that hit the beat instead of sliding off it.

How it works

The core idea is patterns measured in beats. You define pattern strings like 4,4,8,4,4,8 - that's a sequence of shots lasting 4 beats, 4 beats, 8 beats, and so on. The node walks the beats you gave it, chops the song into shots according to your pattern, and computes drift-free frame boundaries for each one. "Drift-free" is the important part: because it maps beats to cumulative sample positions rather than rounding each shot independently, the shots stay locked to the music across the whole track instead of accumulating rounding error.

You get up to four patterns (A, B, C, D) and a pattern_sequence that says which order to play them in, so you can do something like verse-pattern / chorus-pattern / verse-pattern without hand-counting bars. The output is a JSON blob describing every shot, which the pack's FL_Audio_Shot_Iterator and segment-extractor nodes then chew through one shot at a time.

The inputs that matter

• audio - the track, as a standard ComfyUI AUDIO.
• beat_positions - a STRING of where the beats are. This is the catch: the node doesn't detect beats itself. You feed this from FL_Audio_BPM_Analyzer upstream, which does the actual BPM/beat detection and outputs the JSON this expects.
• pattern_A - your primary shot pattern in beats, e.g. 4,4,8,4,4,8. pattern_B/C/D are optional variations.
• pattern_sequence - which patterns to use and in what order (default just A).
• fps - the frame rate the shot boundaries get computed against. Set this to match whatever video model you're feeding, or the frame counts won't line up.

Outputs are sequence_json (the full shot plan) and total_shots (an INT count). The JSON goes to the iterator/extractor nodes; the count is handy for driving loops.

Installing it

Comes with the Fill-Nodes pack. ComfyUI Manager: search ComfyUI_Fill-Nodes, install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/filliptm/ComfyUI_Fill-Nodes

then restart. The audio nodes lean on librosa for analysis, which the pack installs on load - if you see it grinding through dependencies the first time, that's why. And yes, the loud "Machine Delusions" banner in the console is just the pack's startup splash, not a problem.

Where people get tripped up

• You need the BPM analyzer first. This node is step two. Without valid beat_positions from FL_Audio_BPM_Analyzer, it has nothing to cut against. If your shots come out wrong, check the beat detection before touching the patterns.
• fps has to match your renderer. The frame boundaries are computed at the fps you set here. Mismatch it with your actual video pipeline and every shot will be off by a bit.
• It plans, it doesn't generate. Don't expect video out of this node - you get a JSON schedule. The whole point is that the schedule is drift-free, so trust it and let the iterator/extractor nodes do the frame work.
• Patterns are in beats, not seconds. 8 means eight beats, and eight beats is a different length at 90 BPM than at 140. That's deliberate - it's what keeps everything musical.