Audio Frame Transform (Shadertoy)

The name says Shadertoy, but this node isn't a shader - it's the thing that feeds them. It analyzes your audio and packs two measurements into every frame of an IMAGE: the clipped waveform on the top row and the frequency spectrum on the bottom. Each frame is 2 pixels tall and 512 wide, so the whole output is a [frame_count, 2, 512, 3] tensor - a very short, very wide movie of your music's shape over time. It's not a picture you'd hang on a wall; it's a barcode of the audio, and that's the point.

Mechanically it's classic DSP. Audio gets resampled to a fixed 48000 Hz internally - your sample_rate input exists so that resample is correct - then sliced into 2048-sample windows with 512-sample hops. Each window gets a Blackman taper, an FFT, magnitude in dB, and, importantly for smoothness, exponential smoothing (0.8^(60/fps)) so the spectrum decays gently instead of jittering sample-to-sample. Waveform samples are clipped to 0..1; the spectrum is normalized so −100 dB maps to 0 and −30 dB maps to 1. The code credits a well-known soulthreads gist, and you can feel the lineage - it's the same analysis that has powered audio-reactive visuals for years.

Inputs: audio, sample_rate, frame_count, fps. Same quartet as the Beats node, same rules: frame_count and fps define your video timeline, and sample_rate must agree with Load Audio or everything shifts pitch.

Output: IMAGE, [frame_count, 2, 512, 3]. Now the fun part - where does a two-row image go? The intended destination is the pack's own Shadertoy node: feed the spectrum row in as an iChannel texture, write a shader that samples it, and you've got visuals that genuinely react to the music's content rather than just its kick. It's also useful on its own as a per-frame spectral-weight signal for conditioning, the same trick as the Beats node but with 512 frequency bins of detail instead of a yes/no. One catch: there's no built-in row split, so to use just the spectrum you'll crop the bottom row (top row is the waveform) with a standard ImageCrop.

Install is shared across the pack: ComfyUI Manager search "ComfyUI-AudioReactor," or clone, pip install -r requirements.txt, restart. librosa is the dependency that matters; nothing downloads a model, and the analysis runs on CPU.

Troubleshooting is mostly about expectations. A 2×512 frame is not something you eyeball - previewing it gives you a skinny scrolling strip, and that's normal, not a settings bug. If the spectrum looks dead or flat, check the audio is actually loud enough to register above the −100 dB floor, and double-check the loader's sample_rate matches this node's. The biggest real-world friction is the pairing with the Shadertoy node: that node expects a normal IMAGE as a channel, so hand it this output - or a cropped row of it - and let the shader do the sampling.

Reach for this one when you want your visuals to dance with the actual content of the music: not just the beat, but the whole frequency picture.