HeartMuLa Lyrics Transcriber

The HeartMuLa Lyrics Transcriber is the pack's audio-to-text half: feed it any AUDIO and a loaded HeartMuLa Transcription Loader, and it returns the lyrics as a plain STRING. It's Whisper under the hood, tuned toward the music use case - and it gives you a genuinely useful loop that's hard to do anywhere else: transcribe a song, dump the words into the Music Generator's lyrics field, and get your own version of the same lyrics with different music.

It also slots in after generation as a quality check. Generate a track, transcribe it back, and compare - you'll be surprised how often HeartMuLa's own vocals transcribe cleanly, because the model is that good at delivering lyrics. That's the "almost NEVER misses lyrics" reputation from the community, working in your favor.

Inputs that matter

Beyond the transcriptor and audio sockets, the fields are Whisper's familiar knobs:

• max_new_tokens (1–445, default 256) - how much text the model can emit. If long tracks get cut off, raise it.
• num_beams (1–5, default 2) - beam search width. Higher is more accurate and slower; 2 is a sensible default.
• temperature (0–1, default 0) - zero means "greedy, most likely text." The node helpfully maps 0 to Whisper's classic fallback schedule (0.0, 0.1, 0.2, 0.4), which retries at higher temperatures when confidence is low.
• no_speech_threshold (default 0.4) - how sure the model must be that there's speech before it commits. Lower it if the transcriber is writing silence as "thank you" (a classic Whisper failure).
• logprob_threshold (default -1) - the confidence cutoff below which a "speech" segment is dropped. Leave it unless you're getting hallucinations.
• condition_on_prev_tokens (default off) - lets Whisper use its own previous output for context. On is usually more coherent, off is more literal.

Internally it resamples whatever you feed it to 16 kHz mono (Whisper's native rate), runs the transcription, and moves the model back to CPU afterwards.

Install and models

The transcriber needs the HeartTranscriptor-oss model that the Transcription Loader expects:

cd ComfyUI/models/HeartMuLa
git clone https://huggingface.co/HeartMuLa/HeartTranscriptor-oss

And the pack itself:

cd ComfyUI/custom_nodes
git clone https://github.com/Crono141/ComfyUI-JK-HeartMuLa.git
cd ComfyUI-JK-HeartMuLa
pip install -r requirements.txt

Restart ComfyUI and both nodes appear under JK-HeartMuLa.

Gotchas

Whisper's quirks apply: heavy reverb or dense production degrades accuracy, and it can hallucinate words into silence - that's what no_speech_threshold and logprob_threshold exist to rein in. There's no timestamps or speaker labels, so it's a lyrics machine, not a full transcription pipeline. One genuinely useful detail: the audio input is the standard AUDIO type, so it works with Load Audio, the decoder's output, or any other audio node in ComfyUI - you're not locked into this pack's generation path.