Transcribe Lyrics

What it is

Two jobs, one node. First: pull the sung words out of a recording and place them into the score's sections, so a cover can reuse its lyrics or write new ones that fit the same phrasing. Second: take a rendered take plus the lyrics it was given and tell you whether those lyrics were actually sung, section by section.

The second job is the underrated one. When you ask a music model for an instrumental and it hums anyway, or when a verse comes out as mumbling, "listen to the whole thing and judge" is the honest option - and this node automates the measurement part of that.

How it works

The ASR is faster-whisper large-v3, and it runs in a separate process, which is why it doesn't drag the rest of your graph into a transformers-shaped argument. That's also why it's the one optional dependency in the pack. The model is fetched as a Plenio asset on first use (3.1 GB, into models/plenio/asr/faster-whisper-large-v3), and an offline setup will stop and tell you what to place.

Recognition alone isn't enough, because words in a jumble don't help anyone. So if score is connected, the node takes the score's sections as tags and places the transcription on the beat grid - the same timeline that Transcribe Score produced. timeline (optional) is where the bar times and "somebody sings here" markers live. The result is sectioned lyrics you can hand to the writer as a phrasing reference, or edit in the Song Sheet. Low-confidence words are kept but highlighted in the sheet so you can fix them before rendering, since ASR mishears.

The language handling has one trap worth knowing: if brief is a Cover Brief for an original-lyrics cover, its language wins and the language widget is ignored. For new lyrics, the widget describes the source's language instead. device defaults to auto - GPU, falling back to CPU - and the CPU path runs at roughly 0.6× the audio length, so a three-minute song is a two-minute wait. You get a warning when it falls back; you don't get to wonder why it got slow.

Connect expected_lyrics and the node changes job: it becomes the sung-lyrics check, comparing what was heard against what the take was given, with a word error rate per section.

Inputs and outputs

Four required inputs: audio (the source recording, or a rendered take in check mode), engine (the ASR engine picker, currently faster-whisper large-v3), language (auto, or a name or code like English or de) and device (auto / cuda / cpu). Optional: score, timeline, brief, expected_lyrics.

Outputs are lyrics (sectioned draft - or the heard lyrics in check mode), transcript (the plain, unplaced text), language (detected or given, which is what feeds Compose Writing Prompt's lazy language input) and report (words, confidence, placement, or the check result).

Install

# ComfyUI Manager → search "Plenio Music Production System" → Install → restart
# or:
cd ComfyUI/custom_nodes
git clone https://github.com/jplenio/Plenio-Music-Production-System

ComfyUI 0.37.0+, no Python packages from the pack - but this node needs one you install yourself, and it will say so when you first run it:

python -m pip install faster-whisper

faster-whisper is MIT-licensed, and it's the only thing here you have to pip in. The 3.1 GB model file is separate and fetched on first use unless you're offline.

Common issues

• It asks for files or refuses to run offline. The ASR weights aren't there yet, or PLENIO_AUTO_DOWNLOAD=0 is set. Allow the download once, or place the files where the message says.
• Words in the wrong sections. That comes from a missing or stale timeline. Connect the timeline from the same Transcribe Score node that produced the score.
• Nonsense in the transcript. Sung words are harder than speech and everybody's ASR mangles them. That's why the sheet flags unsure words rather than pretending they're fine.
• A check that fails on a take you liked. It reports word error rate, not musical quality. A take that sang an alternate phrasing will score badly and still be the one you want.