Whisper Transcribe

This is the node that actually does the work: hand it a ComfyUI AUDIO and a loaded Whisper model, get back the transcript. Where it earns its place is the timestamps - you get them for free, per recording, in a text you can parse. Where it trips people up is that all three outputs are lists, one entry per audio item, not plain strings.

Why you'd reach for it

Transcription is the inverse of everything else ComfyUI's audio layer does. The community's audio effort goes into making sound - voices, music, foley - and the listening direction is the quiet helper sitting behind it: captioning a dataset you're about to train on, checking that a TTS read actually pronounced what you wrote, or pulling timecodes so a clip can carry subtitles. It's a modest workflow, not a trend.

How it works

The node takes the WHISPER_MODEL from Whisper Loader and runs the pack's native Whisper implementation on ComfyUI's model management and attention path - no openai-whisper package involved.

The pipeline is the classic one. Your audio is converted to a float32 mono waveform and resampled to 16 kHz (stereo or multichannel input is downmixed, not rejected). From there it becomes an 80-bin - 128-bin on large-v3 - log-mel spectrogram, and inference proceeds in 30-second windows, each one re-padded to a full window so the encoder always sees the shape it expects. On the first window with language: auto, the model scores its language tokens and picks the winner for the rest of the recording. Decoding starts at temperature 0 and only escalates when a result looks bad (high compression ratio, low average log-probability) - the standard OpenAI fallback loop. Prompt tokens carry across windows so the transcript stays coherent, and offsets are computed from the start of the recording rather than each window.

Segments whose text is empty, or whose end lands at or before their start, are dropped rather than emitted as blanks.

Inputs and outputs

Beyond whisper_model and audio, there are two settings.

task is transcribe or translate. Transcribe keeps the spoken language; translate forces English output. There's no "translate into French" - the model only ever translates to English, which is a Whisper property, not a limit of this node.

language defaults to auto (detection from the first window) or takes a spoken-language code. Setting it explicitly is worth it: detection on a short or noisy clip is where wrong-language gibberish comes from, and an explicit choice also gets echoed straight back on the language output. Provide a language the checkpoint doesn't know and you get an explicit rejection instead of garbage.

The outputs, all lists:

• text - the transcript, as a single string per recording.
• segments - a JSON array of {start, end, text} for that recording, with times in seconds from the start of the audio.
• language - the spoken-language code actually used for that recording.

Times are approximate; word-level alignment and diarization aren't here, and there's no SRT/VTT formatting. If you need a subtitle file, take segments and build the formatting yourself - the same pack's UC_GetJsonValue node is the obvious way to pull fields out of it.

Installing it

ComfyUI Manager → search ComfyUI-UtilsCollection → Install → restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-UtilsCollection
pip install -r ComfyUI-UtilsCollection/requirements.txt

The one requirement you'll notice if it's missing is tiktoken, which supplies the tokenizer; the pack ships the tokenizer files itself but needs that library to read them. unifiedefficientloader and huggingface_hub come along for the model-loading side. Everything except Whisper keeps working without tiktoken.

Where people get burned

Lists, not strings. Feed one audio file and the outputs look like normal strings, so you wire text into a text node and it works. Feed a batch of eight clips and you now have eight outputs and a text node that silently reads only one. Chain a list-aware or index node if you're processing a folder of clips.

Long audio is slow. A 10-minute recording is roughly twenty 30-second windows, decoded sequentially on your GPU, with a progress bar in the console. If you only need a short line, trim before you transcribe.

translate with a non-English language. That means "this audio is German, write it in English" - the transcript comes back English, and people seeing a German language output assume it broke.

Missing tiktoken. You get Whisper requires tiktoken. Install tiktoken in ComfyUI's Python environment and restart ComfyUI. Install it into the same interpreter that runs ComfyUI, not your system Python.