Cover song · source lyrics / phrasing (Whisper)

Covering an existing track means dealing with its words, and SheetSage2 - the model that reads a musical score out of your audio - doesn't transcribe lyrics. It extracts music. So if you want the original words in your cover, or you want new words that fit the original phrasing, something has to listen to the vocal and write down what it heard.

That's this node, and it's the toolkit's longest-running piece of machinery: Whisper, run over your source song, producing timed phrases that either guide the LLM's new lyrics or become the authoritative text of the old ones.

It's gated, and that matters

This node is only active for a YuE2 Cover run with the cover lyrics mode set to new lyrics or original lyrics. Anything else - an instrumental cover, a plain YuE2 song, MiniMax Music 3 - returns an empty result and its report says inactive or not_requested rather than loading a checkpoint of any kind.

That's deliberate, not laziness. Whisper on a clean instrumental buys you nothing but a hallucinated word list, and the toolkit refuses to spend the time or the VRAM. So don't read an empty output as a broken node; read lyrics_report_json first.

The inputs you'll actually touch

Three of the required inputs are wiring, not settings: the song-model profile, the cover source (which carries the lyrics mode and the audio file), and the model check report - that last one exists so configured downloads finish before Whisper tries to load anything.

Then the engine:

• whisper_model - a CTranslate2 checkpoint from the pack's catalog. Default and currently only bundled choice is whisper-large-v3; add a catalog entry to offer another.
• language - auto or one of 29 codes. Auto detection is usually right, and pinning the language actually being sung is the documented fix for wrong-language or repeated output. This is the source recording's language, not the language you want the new lyrics in.
• device / compute_type - auto prefers CUDA and drops to CPU with int8. float16 on CUDA is the recommended default on a 16 GB card.
• vad_filter - off by default, and it should stay off for songs. Speech detection misses singing.
• beam_size - 5 by default. Higher hears slightly more, slower.
• condition_on_previous_text - off, because carrying text across chunks is how you get a line repeated into the next section.

Outputs are cover_lyrics (plain text, useful if you're building a custom graph) and lyrics_report_json, which carries the text plus segment and word timestamps and the engine settings that actually ran. The bundled workflow feeds the report to the Structured Song Prompt and the parser so the timing evidence survives into the production record; connect that one, not just the text.

Why it's built like a small infrastructure project

Transcription on a long song is the slowest thing in the run, so the engine runs in a separate, cancellable process with progress every 15 seconds. A stalled GPU worker times out after 180 seconds without progress, CPU after 600, and the total budget is the larger of 20 minutes or 20 times the source duration. A CUDA runtime failure or timeout retries once on CPU/int8; later auto runs stay on CPU for that ComfyUI session until you restart. Cancelling discards partial text - no half transcript ever silently becomes your lyrics.

Install

The pack first:

cd ComfyUI/custom_nodes
git clone https://github.com/jplenio/ComfyUI-MiniMax-Music-Production-Toolkit.git
cd ComfyUI-MiniMax-Music-Production-Toolkit
python -m pip install -r requirements.txt

faster-whisper is in that file (on an older install: python -m pip install -r requirements-whisper.txt), pulling CTranslate2, PyAV, ONNX Runtime and Tokenizers without touching your PyTorch. The checkpoint is separate: keep whisper_models and auto_download on in the model check node, and a text-bearing cover run downloads the pinned large-v3 files - about 2.9 GB - into models/audio_encoders/whisper-large-v3. The folder name and the whisper_model value have to match.

Where it goes wrong

VAD ate the song. Older workflows shipped with voice detection on, and on singing it can retain 7 seconds of a 207-second track. Current defaults are off; if a report shows retention under half the audio, the node retries the whole file without VAD and records both attempts.

Empty or fragmentary text. The node has a heuristic guard: on audio of 60 seconds or more, fewer than 12 words ending before 20% of the song stops the run with guidance, because that's a hallucinated opening rather than lyrics. It's a heuristic, not a certificate.

Words that aren't there. Music makes ASR invent things. Pin the language, keep condition_on_previous_text off, and read the transcript before you build a whole cover on top of it. Auto speech recognition on sung material is a good first draft, never verified ground truth - and if the transcription is wrong, the fix is to hand reviewed text to the prompt and parser's cover_lyrics inputs in a custom graph.