Transcribe Lyrics (Whisper)

Whisper transcription has been a command-line staple forever, but getting it inside a ComfyUI graph - with the audio coming from your video branch and the text coming out on a socket - is exactly what this node does. Transcribe Lyrics takes any AUDIO input and returns four outputs: plain text, LRC-timed lrc lyrics, segments_json (start/end/text records), and the duration in seconds.

The "lyrics" in the name isn't just flavor. It's built to feed music-video pipelines - this pack's H3 nodes want sung words delivered as timed blocks, and Whisper with LRC output is how you get the timing without typing it by hand.

How it works

Pick a model from the required inputs: openai/whisper-large-v3 (default, most accurate), large-v3-turbo (much faster), medium, small, base - or drop any Whisper folder into ComfyUI/models/whisper/ and select it as local:<name>. Models download from Hugging Face on first use.

Then a few knobs that matter more than they look:

• language - default auto lets Whisper detect it, but the tooltip flags the honest problem: detection is less reliable on sung vocals. If you're transcribing music and the words come out garbled, set the language explicitly.
• task - transcribe keeps the source language; translate outputs English.
• beam_size - 1 is greedy and fast (default); 5 is more accurate and slower. For lyrics, bumping to 5 is usually worth it.
• precision - auto picks fp16 on CUDA, fp32 on CPU. If you see NaN or empty output, switch to fp32.
• keep_model_loaded - on by default, keeps Whisper in VRAM between runs; turn off to free it after each transcription.
• hint_prompt - biases Whisper toward names or spellings you supply. One gotcha: on audio under 30 seconds, a prompt disables timestamps - an upstream transformers limitation, not a bug here.

Audio over 30 seconds switches to transformers' sequential long-form decoding, so timestamps stay absolute and words aren't cut at chunk boundaries. That's the detail that separates this from a naive "chunk it and hope" implementation.

Outputs and how to use them

• text - the straight transcription
• lrc - LRC-formatted lyrics with [MM:SS.mm] timestamps, ready for a karaoke-style overlay or a timed captioner
• segments_json - machine-readable records, for when you want to filter, re-timestamp, or drive something programmatic
• duration - seconds, so downstream duration math doesn't have to guess

In the music-video context this pack cares about, lrc or segments_json is how the sung words reach the H3 prompt nodes with real timing instead of a wall of un-timed text.

Gotchas

The obvious one: sung vocals are harder for Whisper than speech, full stop. Set language explicitly, raise beam_size, and use hint_prompt for artist or song-specific spellings. And remember keep_model_loaded - if you run this next to a diffusion model on one GPU, the resident Whisper model is eating VRAM you might want back; that's what the pack's Unload MiniCPM-style cleanup pattern is for, except here the toggle is right on the node.

Install

Part of TrentNodes (TrentHunter82/TrentNodes):

# ComfyUI Manager: search "Trent Nodes"

# or:
cd ComfyUI/custom_nodes
git clone https://github.com/TrentHunter82/TrentNodes.git
cd TrentNodes && pip install -r requirements.txt

It needs transformers (the pack's requirements install it), and Whisper weights download on first use - a large-v3 is a real download, so the first run isn't instant. Logic is covered by the pack's tests/test_transcribe_lyrics.py, which is a good sign for a node this fiddly.