WhisperX Node

If you've ever needed to caption a video and ended up on some web tool that holds your transcript hostage behind a paywall, this is the fix. WhisperX Node is the heart of AIFSH's ComfyUI-WhisperX pack: drop in an audio file (or a video - it decodes the audio track), and it transcribes, word-level aligns, writes a proper .srt file to your ComfyUI output folder, and can optionally translate every line or tag who said what. No API, no key, nothing leaving your machine - except when you opt into speaker labels.

It's the subtitling half of a video pipeline that ComfyUI otherwise can't do natively. The image and video generation side is well covered by the ecosystem; timed transcription is not, which is exactly why people reach for this pack.

How it works

Under the hood it's m-bain's WhisperX, which takes OpenAI's Whisper and adds the thing stock Whisper lacks: word-level forced alignment. A plain Whisper transcript puts timestamps at chunk level, so subtitles drift and land mid-word. WhisperX transcribes with batched faster-whisper, then runs a dedicated alignment model for the detected language so each word's timestamp lines up. That's the difference between subtitles that feel synced and ones that feel close enough.

The flow in this node: transcribe in batches → load the per-language alignment model → align → optionally run pyannote diarization and assign speaker labels → compose the lines with the srt library → write two files into ComfyUI/output/. Compute is float16, and the node picks CUDA or CPU based on ComfyUI's cuda_malloc support check. The models load fresh on every run, so the first execution also downloads weights from HuggingFace.

The inputs that actually matter

There are eight, but you'll touch maybe four:

• audio - an AUDIOPATH from the pack's LoadAudioVideoPath node (or any other node that emits that type). This is the only way in.
• model_type - default large-v3, with distil-large-v3, large-v2, and large-v3-turbo. Large-v3 is the accuracy pick but a big download and slow; large-v3-turbo is the one I'd actually reach for on a consumer GPU. Worth knowing: the node silently remaps it to a CTranslate2 build (deepdml/faster-whisper-large-v3-turbo-ct2), which is why it runs fast.
• batch_size - default 4. How many audio chunks transcribe at once. Crank it up on a strong GPU, drop it if you're VRAM-poor.
• if_mutiple_speaker - off by default, and the name is a typo in the code. Flip it on and you get speaker labels via pyannote, but only if use_auth_token is a real HuggingFace token (see troubleshooting below).

The rest are translation: if_translate, translator (37 free-ish web engines, default alibaba), and to_language (11 targets, default en). The translation runs per subtitle line through the translators library - those are public web APIs, no key needed for most engines.

Outputs

Two, both of type SRT (a path to a .srt file): ori_SRT and trans_SRT. With translation off, both point at the same original file, so don't panic. When speaker labels are on, each line gets its speaker prefixed, e.g. SPEAKER_00. Wire them into the pack's PreViewSRT to read them on the canvas, or SRTToString to get the text as a string - or just grab the files from ComfyUI/output/.

Installing

Via ComfyUI Manager, search ComfyUI-WhisperX. Or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/AIFSH/ComfyUI-WhisperX.git
cd ComfyUI-WhisperX
pip install -r requirements.txt

Then restart ComfyUI. Two things will bite you:

• ffmpeg must be on your command line. The vendored WhisperX decodes audio by shelling out to the ffmpeg CLI - Python libs alone won't save you. apt install ffmpeg on Linux, or install it via WingetUI on Windows, exactly as the README says.
• Weights auto-download from HuggingFace - large-v3 is ~3 GB. If you're in China and HF is unreachable, the README points at hf-mirror env config.

Troubleshooting

The classic failure is a RuntimeError "Failed to load audio" - that's ffmpeg missing or unable to decode the file. Fix that first. If speaker diarization 403s or silently does nothing: the default token placeholder is a string, not a token - you must (1) accept the pyannote/segmentation-3.0 and pyannote/speaker-diarization-3.1 gated model licenses on HuggingFace, (2) create a read token, and (3) paste it into use_auth_token. No way around it. And if transcription crawls, check whether the node fell back to CPU - if it did, that's why.