Audio Beats & Notes

Music information retrieval sounds like a whole field of study - and it is - but what you actually want is "where are the beats in this song?" Audio Beats & Notes (the display name for AudioMIRStage) answers that: it analyzes a track and hands you back timestamped keyframes for every beat, onset, or note, plus a set of labels. Then you wire those into the pack's audio-reactive machinery and your video pulses along with the music.

This is the node that makes a generated clip feel choreographed. You'll use it for the classic moves: keyframe a zoom or a scale change on every beat, flash an overlay on each onset, sync a text reveal to a snare hit, or rotate/translate elements to the rhythm. The source shows it's a self-contained numpy implementation - onset detection on an energy-based frame difference, a beat tracker, and a simple note extractor - so there are no models, no tempo-pedestal-of-models, just math on the decoded audio.

How it works

The mode combo picks what you're detecting - beats (the musical grid, with an estimated BPM), onsets (every transient, more dense and percussion-friendly), or notes. The controls:

• threshold (0.05–1, default 0.3) - how strong a transient must be to count as an event. Lower = more sensitive, catches ghost notes and noise.
• min_gap_s (0.01–1, default 0.05) - minimum gap between events, so one drum hit doesn't get double-counted.
• field (v, scale, opacity, x, y, rotation) - the parameter the keyframes are pre-targeted for, so downstream audio-reactive stages know which property to drive.

Feed audio (COMFYTV_AUDIO) or video (its soundtrack). The outputs are the interesting part: keyframes (a COMFYTV_TEXT payload of timestamped events, ready for the parameter-automation stages) and labels (also COMFYTV_TEXT - the human-readable list, like "beat 1, beat 2, …" or the note names). Internal inputs (force_run_token, project_id, parent_output_id) are frontend plumbing.

Installing ComfyTV

cd ComfyUI/custom_nodes
git clone https://github.com/jtydhr88/ComfyTV

Full backend restart, then ComfyTV → AudioFX; ComfyUI Manager finds "ComfyTV". Zero extra Python deps - the analysis is pure numpy on decoded audio, already bundled. No models.

The pack-wide install trap: on macOS / ComfyUI Desktop / multi-install machines, cd ComfyUI/custom_nodes can clone into the wrong instance - clone succeeds, stages never appear. Find the running instance's path in the startup log, clone into that instance's custom_nodes by absolute path (quote paths with spaces/parentheses), confirm ComfyTV/__init__.py is top-level, restart the backend fully.

Common issues

• "needs an audio (or video with audio) input." That's the author's own error - the stage has nothing to analyze. Connect the track first.
• Beats are off by a hair. Detection on a mix is never sample-perfect. threshold tuning helps; for drums use onsets, for song structure use beats. If it's consistently late, nudge your downstream keyframes - that's the workflow's job, not the detector's.
• Way too many or too few events. Too many: raise threshold and min_gap_s. Too few: lower threshold. There's a sweet spot per track and it's cheap to find by running it a couple of times.
• The field choice seems to do nothing here. It doesn't here - this stage just labels the keyframes for the target field. The actual animation happens when those keyframes drive the audio-reactive parameter automation stages.

It's the closest thing to a free choreographer in ComfyUI. Analyze once, drive everything.