FL Audio Beat Prompt Schedule

This is the flagship of Fill-Nodes' audio-reactive workflow, and honestly it's the node you build a whole music video around. FL Audio Beat Prompt Schedule loads an audio file, detects its beats, and gives you a full-screen editor where you drag prompt clips onto a timeline aligned to the waveform. Each clip is a prompt that's "on" for a frame range, with fades and crossfades, and the whole thing comes out the other end as a typed schedule your video generator can actually consume. You write "the subject slowly turns toward camera" for bars 1–2, "camera pushes in on the beat" for bars 3–4, and the model gets a per-frame prompt mask instead of one flat sentence.

It's built for MiniMax H3, the omni-modal video model that's native at audio. The schedule you build here feeds the FL MiniMax H3 Beat Shot Planner - which lives in the separate ComfyUI-FL-MiniMaxH3 pack, not this one (node IDs are unchanged, so old workflows keep loading once the new pack is installed).

How it works

Beat timing comes from beat-this, an open-source (MIT) beat tracker. The first analysis downloads its 77.3 MiB final0 checkpoint to ComfyUI/models/beat_this/beat_this-final0.ckpt, SHA-256-verifies it, and releases it from GPU memory when done. Analysis covers and caches the full source by content hash, so trimming the crop or nudging the beat offset reuses the cached analysis instead of re-scanning. Everything loads without queueing - pick your audio and the waveform and beat markers appear immediately.

The editor is where the magic is: a waveform with beat, downbeat, transient, and drum marker lanes, a frame ruler you drag prompt clips onto, right-click to set audio In/Out points, and a shared boundary between touching clips you can drag to lengthen one and shorten the other (it snaps to the beat grid). Three reactive-envelope slots below each pick a source (beat grid, downbeat, kick, snare, hi-hat…) with stride, phase, attack/hold/release, and floor/peak controls. The schedule text format is human-readable too:

[0 - 48 | fade_in=6 | fade_out=6]
The subject slowly turns toward camera.

[48 - 96 | fade_out=6 | crossfade=12]
The camera pushes forward on the beat.

Frame ranges are zero-based with exclusive ends. fade_in/fade_out blend with the global prompt; crossfade blends a clip directly into the previous one.

Inputs and outputs that matter

You'll set: timeline (the schedule above), fps (24 for H3), audio_file (upload or pick - waveform loads without queueing), and optionally beat_positions from FL Audio BPM Analyzer to override the internal detection. The sequencer-owned fields (render_groups, envelope_layers, analysis_cache_key) are managed by the popup editor - leave them alone.

Outputs: prompt_schedule (the resolved schedule for compatible FL diffusion nodes), total_frames, audio (the frame-aligned crop, AUDIO), BPM, three FL_AUDIO_ENVELOPE outputs (envelope_1/2/3) for visualizers and post-effects, and prompt_envelopes (FL_PROMPT_ENVELOPE_SET) for the Beat Shot Planner.

Install

It ships in ComfyUI_Fill-Nodes - Manager search "ComfyUI_Fill-Nodes", or:

cd ComfyUI/custom_nodes
git clone https://github.com/filliptm/ComfyUI_Fill-Nodes

then restart. Heavy pack, heavy requirements (librosa, beat-this, imageio[ffmpeg], opencv-python, and much more) - the first install takes a while. Stem separation and Whisper transcription are explicit button actions, never automatic; the Whisper model only downloads to ComfyUI/models/whisper/ when you click Transcribe.

Gotchas

The big one is cost, not setup: strict scheduling evaluates H3 once per unique active prompt on every sampling step. Start with a few broad sections, and disable cross-step caching (or model compilation) while you validate a masked workflow - you don't want to wait through a whole generation to discover a typo. And remember beat detection needs the model checkpoint on first run; if the first analysis sits there downloading, that's normal, not a hang.