Nodes/ComfyUI-AudioScheduler/AudioData to FFTs
ComfyUI Node

AudioData to FFTs

Chop your song into frames and get a spectrum per frame

By a1lazydog·Created 3 years ago·Updated 2 years ago· 112
AudioData to FFTs
  • audio
  • AUDIO_FFT
  • total_frames
channel0
frames_per_second12
start_at_frame0
limit_frames0

This is where audio stops being a continuous waveform and becomes discrete keyframes. AudioToFFTs takes the AUDIO_DATA from AudioToAudioData, slices the track into fixed-length frames, runs an FFT on each one, and hands you a list of AUDIO_FFT objects - one spectrum per frame - plus an INT telling you how many frames you ended up with.

Why frames? Because the whole point of this pack is to drive an animation, and animations are driven frame by frame. Twelve frames per second means twelve values per second of music to steer whatever you're generating. If your animation runs at a different rate, you adjust here, not downstream.

The inputs that matter

  • frames_per_second (default 12) - the big one. It sets how many FFT frames the track is divided into, and therefore how many keyframes every downstream node produces. 12 matches a typical AnimateDiff frame rate; crank it to 24 for smoother curves at the cost of more compute. The total_frames INT output is your friend for sanity-checking this.
  • channel (default 0) - which audio channel to analyze. This is where the mono-ish handling from AudioToAudioData keeps things simple: you'll basically always use 0.
  • start_at_frame (optional, default 0) - skip ahead. A negative value offsets from the end of the track, so -10 starts ten frames before the track ends. Handy for looping.
  • limit_frames (optional, default 0) - cap how many frames you get. 0 means "everything." If you set it, total_frames reflects the cap.

Outputs: AUDIO_FFT (the list) and total_frames (INT).

Gotchas

The node validates its inputs and will throw rather than silently do something dumb: frames_per_second must be positive, and the absolute value of start_at_frame can't exceed the total frame count. Both are real errors you can hit by typing a stray zero.

Also worth knowing before you start: this is pure extraction, no model involved. It's the same FFT math a scipy.fft.fft call does per frame, so it's fast and costs essentially zero VRAM - the only heaviness in an audio-reactive workflow comes from the actual video generation downstream. And a small design note: AudioToFFTs expects AUDIO_DATA, so it must be fed by AudioToAudioData, not directly by LoadAudio. Some early example workflows skip that hop and won't load in the current pack; drop the adapter node in and they'll run.

CategoryAudioScheduler

Inputs (5)

NameTypeDefaultDescription
audioAUDIO_DATA
channelINT00–24
frames_per_secondINT120–240
start_at_frameoptINT0-100000–100000
limit_framesoptINT00–100000

Outputs (2)

NameTypeDescription
AUDIO_FFTAUDIO_FFT
total_framesINT