Apply Whisper

This is the node that does the actual listening. Everything else in ComfyUI-Whisper - burning text onto frames, exporting SRT files - is downstream of what Apply Whisper produces: a transcript with timestamps, down to the individual word. Feed it audio, get back text plus two flavors of timing data. If you've ever manually typed out captions for a video, this is the node that makes you never want to do that again.

What it's doing under the hood

Apply Whisper wraps OpenAI's Whisper speech-to-text model - the actual openai-whisper package, not a lighter reimplementation - and runs it against whatever audio you hand it. Whisper was trained on a huge amount of multilingual audio and it's genuinely good at this: accents, background noise, and mixed-language clips all degrade gracefully rather than falling over. The node's real value-add over calling Whisper yourself is that it doesn't just give you a transcript - it aligns that transcript to time, at both the segment level (sentence-ish chunks) and the word level (every single word gets a start and end timestamp). That word-level alignment is what lets the subtitle nodes downstream place text precisely, and what lets Save SRT produce a properly-timed subtitle file instead of one giant caption sitting on screen for the whole clip.

The inputs and outputs that matter

Only two things are required:

• audio - a ComfyUI AUDIO input. Load it with a standard audio-loading node, or pull it out of a video with something like VideoHelperSuite's load-video-with-audio node.
• model - which Whisper checkpoint to run: tiny.en/tiny through medium.en/medium, the large family (v1, v2, v3), plus large-v3-turbo and turbo. The .en variants are smaller and faster if your audio is English-only; the multilingual ones handle everything else. turbo is the sweet spot for most people - most of large-v3's accuracy at a fraction of the compute cost, which matters because Whisper inference isn't instant.

Two optional fields worth knowing: language (default auto, with 100+ languages to pin manually if auto-detect guesses wrong on short or noisy clips) and prompt - free text that biases the transcription, useful for coaxing Whisper to spell a proper noun or unusual term correctly instead of phonetically guessing at it.

Three outputs: text (plain transcript, no timing), segments_alignment, and words_alignment. Both alignment outputs share the pack's own whisper_alignment type, but they're not interchangeable in practice - segment-level groups words into sentence-sized chunks (what Save SRT wants, since nobody wants word-by-word flashing SRT cues), while word-level is per-word timing (what Add Subtitles To Frames and Add Subtitles To Background want, since they place and time individual words on screen). Wire the right one to the right downstream node.

How to install it

Easiest path: open ComfyUI Manager, search ComfyUI Whisper, install, restart. Manual route if you'd rather:

cd ComfyUI/custom_nodes
git clone https://github.com/yuvraj108c/ComfyUI-Whisper

then restart ComfyUI. The pack's dependencies are openai-whisper, pillow, and soundfile - nothing exotic, but openai-whisper is a real chunk of install weight on its own. You don't need to manually fetch a model file: the first time you run a workflow with a given model selected, Whisper downloads that checkpoint automatically into ComfyUI/models/stt/whisper. The large variants are multiple gigabytes, so pick tiny or base first just to confirm your graph is wired right before committing to a slow first download.

Common issues & troubleshooting

First run on a given model is slow, and it looks like it's hanging. It's not - it's downloading the checkpoint. Check your ComfyUI console log for download progress before assuming something broke.

"No such file or directory: ffmpeg" or audio loading fails outright. openai-whisper shells out to ffmpeg to decode audio, and unlike its Python dependencies, ffmpeg isn't something pip can install for you - it has to be on your system PATH. This trips up a lot of people because everything looks fine until the actual audio decode step. Install ffmpeg through your OS package manager (or make sure your ComfyUI environment already ships it, which most cloud/managed setups do).

Auto-detected language is wrong on a short clip. Whisper's language detection runs on the first ~30 seconds; a short clip, heavy accent, or code-switching audio can fool it. Set language explicitly instead of trusting auto once you notice this happening.

Transcript misspells a name or term every time. That's exactly what prompt is for - put the correct spelling in the prompt field to bias Whisper toward it, rather than fighting the same wrong transcription run after run.