SignalToCurve

This is the node that actually makes dmarx's ComfyUI-AudioReactive pack do something. Threshold and the other ops are mostly unfinished scaffolding, but SignalToCurve works, and it's the whole reason the pack exists: it's the bridge that turns audio into something your animation pipeline can drive parameters with.

What it is

Audio-reactive animation is a simple idea with a plumbing problem. You want the beat to push your KSampler denoise up, your ControlNet strength to swell on a snare, your camera to swing with the bass. That means some part of your graph has to turn audio - a big array of samples - into parameter values over time. The community's established route for that is dmarx's keyframed curve ecosystem (the same author's ComfyUI-Keyframed pack). SignalToCurve is the junction: it takes this pack's SIGNAL and emits a KEYFRAMED_CURVE, which is exactly what keyframed-based nodes consume.

It's the ComfyUI port of the audio-to-curve step from dmarx's older "Video Killed the Radio Star" Colab notebook - the 2022 project that automated whole music videos from one track. The README says it in a single line: "porting audioreactivity pipeline from vktrs to comfyui."

How it works

A SIGNAL in this pack is a dict: y (the sample array), sr (sample rate), and an is_raw flag. SignalToCurve pulls y and sr out, then builds a kf.Curve with every sample (or every frame) as a keyframe, labeled with the signal name. The one subtlety is time:

if the signal is raw (actual samples), each sample gets a timestamp in seconds via samples_to_time
if it's already a feature (like RMS or novelty, which are frame-based), it uses frames_to_time instead

Either way you get a curve where time is the x-axis and amplitude is the value - the exact shape you need to feed a node that interpolates a parameter across your animation.

Inputs and outputs

One input, one output, nothing to configure:

signal - a SIGNAL from any audio node in this pack (or ARReadAudio's loaded file)
KEYFRAMED_CURVE - a kf.Curve you wire into keyframed/ComfyUI-Keyframed nodes to drive anything they'll take a curve for

The input is forced (forceInput: true), so you can't accidentally wire a constant in.

Where people get burned

The gap between "raw" and "feature" matters more than it looks. A three-minute song at 44.1 kHz is about eight million samples - and every sample becomes a keyframe. Feed SignalToCurve the raw waveform and you're asking the keyframed library to interpolate millions of points. That's why the intended pipeline is: run an op first (OpRms, OpNovelty, OpSmooth) to boil the audio down to a manageable feature, then convert. The is_raw flag the ops carry is how the node knows which time axis to use.

Installing it

Same install as the whole pack. ComfyUI Manager (search "ComfyUI-AudioReactive"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/dmarx/ComfyUI-AudioReactive
# restart ComfyUI

On load it pip-installs scipy, scikit-learn, librosa, and loguru automatically. The trap: keyframed is not auto-installed, but it's imported at startup - so without it the pack fails to load entirely. You need dmarx's ComfyUI-Keyframed pack (or pip install keyframed) for SignalToCurve to exist at all.

Bottom line

If you want music-video-style animation where the track drives the motion, SignalToCurve is the piece that connects audio-land to keyframe-land. It's a work-in-progress pack - last touched January 2024, some ops still don't run - but this node is solid. Feed it a smoothed feature, not the raw mix, and it'll do exactly what it says on the tin.