Audio Analysis

Audio Analysis is the heart of ComfyUI_Yvann-Nodes, yvann-ba's audio-reactive pack. Every video you've seen where the image pulses, cuts, or warps exactly on the beat starts here: this node turns a song into a per-frame number that everything downstream can react to.

What it actually does

You give it an audio file, a frame count, and a frame rate, and it measures the loudness of the music in each frame of your animation. The output audio_weights is one float per frame - the classic "beat" curve you can wire into IPAdapter, ControlNet, AnimateDiff, or prompt scheduling.

The mechanism is simple and CPU-friendly: the waveform is chopped into batch_size frames (each frame is batch_size / fps seconds of audio), and each frame gets an RMS energy value - root-mean-square loudness. Those values are normalized to 0–1, everything below threshold is zeroed out, and multiply amplifies whatever's left (clamped at 1). That's it. No API, no key, no hidden server.

Where it gets clever is analysis_mode. Pick Drums Only, Vocals Only, Bass Only, or Others Audio and the node first runs an audio separation model to isolate that stem, then measures its loudness per frame. Want the animation to pump with the kick drum? Drums Only gives you a weight curve that spikes on the kick, not the vocals. Full Audio skips separation entirely and just gives you the overall envelope - handy for a quick test without waiting on the model.

The inputs that matter

• audio_sep_model - from Load Audio Separation Model. Required even in Full Audio mode, which is a mild annoyance, but just wire it up.
• audio - the actual track. The pack doesn't ship an audio loader; the example workflows feed this from Video Helper Suite's Load Audio (VHS_LoadAudioUpload). Anything producing the AUDIO type works.
• batch_size - how many frames of weights you want. Set it to match your animation length. Set it to 0 and it's computed from duration × fps automatically.
• fps - your target frame rate; combined with batch_size it decides how many audio samples land in each frame.
• threshold (default 0.5) and multiply (default 1) - threshold kills quiet sections, multiply pumps the loud ones. Raise threshold if the curve is too noisy, raise multiply if it's too flat.

Outputs

• audio_weights - the one you care about. A FLOATS list, one value per frame, 0–1. Feed it into Audio Peaks Detection, Edit Audio Weights, or Floats To Weights Strategy.
• processed_audio - the isolated stem (or the input, in Full Audio mode), resampled back to the original rate.
• original_audio - your input, untouched.
• graph_audio - a matplotlib preview of the weight curve, so you can see what you got before you build the whole graph around it.

Gotchas

First run with any mode except Full Audio downloads the separation model - Hybrid Demucs is pulled from torchaudio's hub, Open-Unmix's weights from HuggingFace. Give the first queue a minute. And the README claims models land in ComfyUI/models/audio_separation_model/, but the code actually caches Open-Unmix at ComfyUI/models/openunmix/ - don't go hunting in the folder the README names.

The separation itself runs on your GPU (it's a real neural network), so if you're on a low-VRAM card, Full Audio mode is your friend until you need the stems.

Install

ComfyUI Manager → search ComfyUI_Yvann-Nodes → install → restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/yvann-ba/ComfyUI_Yvann-Nodes
# restart ComfyUI

Expect a chunky install - requirements.txt pulls torchaudio and openunmix, and openunmix drags in the librosa/soundfile stack with it.

Troubleshooting

Community threads on this pack (the author posts on r/StableDiffusion as Glass-Caterpillar-70) report two recurring pains: workflows that throw "TypeError: input is null" on load - almost always a missing or broken input wire, so rebuild the audio→batch_size→fps links - and a general "I don't understand what creates what effect" learning curve. That's what graph_audio is for: preview the weights before you wire them anywhere, and you'll know exactly what the song is going to do to your animation.