Difforum · Audio Analyzer

Audio-reactive diffusion usually means a plugin that pretends to "feel" the music and gives you one slider. This node does the unglamorous version: it reads your track, chops it into one value per frame, and hands you six bands you can plug anywhere a number goes.

What comes out

The output is a DIFFORUM_AUDIO bundle containing, per frame, amp (overall loudness), low, mid, high (roughly 20–250 Hz, 250–2000 Hz, and up), onset (spectral flux - attacks and hits) and beat (onset peaks gated above an adaptive threshold, 0 or 1 with a short decay tail). All normalised to 0–1.

Under the hood it's a plain numpy FFT over the track, resampled to your frame count. No torchaudio, no librosa, no model. That's why it installs cleanly on Python 3.12 while the original Deforum ecosystem doesn't.

It also gives you bands_plot - an IMAGE with the six curves drawn, which is much more useful than it sounds. You look at it once and immediately know whether your beat detection is working or your track is just a wall of low end.

The four settings

• smoothing (0.25) - an envelope over the bands. 0 keeps every transient, 0.5+ is a slow swell. The tooltip's advice is right: 0.2–0.3 reads well on camera, because a camera that jitters at every hi-hat looks broken rather than reactive.
• beat_sensitivity (1.5) - the threshold is the mean plus this many standard deviations of the onset signal, so higher means fewer, stronger beats detected. Turn it up on dense electronic music and down on sparse acoustic stuff.
• offset_seconds (0) - start reading the track here. This is the sync-with-an-edit dial: if your hit is at 0:42 in the edit and at 0:00 in the file, this is how you line them up without re-rendering the audio.
• params - the Setup wire. The node needs to know how many frames you're making, and it doesn't guess.

Where the values go

The DIFFORUM_AUDIO output feeds three places, and you can use all three at once. Into Camera (expressions) or Camera (keys) as the audio input, so expressions can reference low, beat and friends directly - 0:(beat*45) is a rotation that whips on every hit. Into a Director camera block's reaction, which is the no-math way to get beat-pulsed speed or bass-driven zoom. And into Audio Curve, which converts one band into a proper 0–1 schedule for any socket that takes a DIFFORUM_SCHEDULE.

Install

ComfyUI Manager → search "Difforum", or:

cd ComfyUI/custom_nodes && git clone https://github.com/chillithebillis/Difforum.git difforum

Restart. requirements.txt is numpy; audio decoding comes from ComfyUI's own Load Audio node, so there's nothing extra to install here.

Where people get burned

Two classics. First, forgetting the Setup wire - the node will tell you it needs params rather than quietly analysing the first 120 frames. Second, expecting amp to equal "how exciting this moment is". amp is loudness, and loudness is usually the least musical of the six. For camera moves, low gives you the pumping feel and beat gives you the hits; onset sits between them and is the one to reach for when the low end is muddy.

If your beat curve comes back as a flat line, sensitivity is too high for the track - check bands_plot before you rewrite your whole graph around it.