Whisper Auto Adjust ⚡🅡🅞🅣🅘

This node belongs to the Whisper corner of RyanOnTheInside's Flex Features system, which exists to do one thing: turn spoken words into a numeric signal you can drive anything else in the pack with - a mask, a particle emitter, an IPAdapter weight. But word-synced effects only look right if the timing is right, and OpenAI's Whisper is known for word-level timestamps that drift a bit - often a couple hundred milliseconds - especially right at the start of a word or coming out of silence. WhisperAutoAdjust re-checks Whisper's timing against the actual audio waveform and nudges it to match where speech genuinely starts.

One thing worth being upfront about: this pack doesn't transcribe audio itself. alignment_data - the input every node in this Whisper family needs - comes from a separate, commonly-used pack called ComfyUI-Whisper. You need that installed and run first to get anything to feed in here.

How it works

It takes alignment_data plus the original audio, then runs energy-based speech detection: it scans a sliding detection_window of samples, compares the energy in that window against energy_threshold, and finds where speech actually kicks in versus where Whisper's alignment claimed it did. Bigger detection_window means steadier, less noisy detection at the cost of some precision; smaller means sharper timing but more sensitivity to noise. Lower energy_threshold catches quiet or mumbled speech; raise it and only clearly loud speech registers.

The inputs and outputs that matter

detection_window (256–8192, default 1024) and energy_threshold (0.00001–0.001, default 0.0001) are the two knobs, and they trade off against each other in the usual detection-tuning way - tighten one, loosen the other, depending on how clean your source audio is. alignment_data (type whisper_alignment) and audio are both required inputs. The single output is whisper_alignment - the corrected version, ready to feed into WhisperFeature or WhisperToPromptTravel.

How to install it

Via ComfyUI Manager, search "RyanOnTheInside." Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/ryanontheinside/ComfyUI_RyanOnTheInside.git
cd ComfyUI_RyanOnTheInside
pip install -r requirements.txt

Restart ComfyUI. And separately, install ComfyUI-Whisper (also available through ComfyUI Manager) if you don't already have it - that's the pack that actually runs Whisper and produces the whisper_alignment data this whole family consumes. RyanOnTheInside's Whisper nodes are consumers, not a transcription engine of their own.

Common issues & troubleshooting

If energy_threshold is set too low, background noise or room tone in a noisy recording gets misread as speech and the adjustment goes wrong in the opposite direction from what you wanted. Set it too high and quiet speakers or soft consonants get skipped entirely, leaving those words un-adjusted while louder ones shift correctly - which shows up as some words syncing better and others not moving at all.

If sync still looks off after running this, that's not necessarily a sign it's broken - automatic energy detection is a best guess, and it can guess wrong on a particular clip. That's what its sibling WhisperTimeAdjuster is for: a manual, constant offset you apply on top when the automatic pass doesn't get you all the way there.