HeartTranscriptor (Lyrics ASR) 🎤

Ever want to grab the exact words out of a song and drop them into a workflow? That's what this node does. HeartTranscriptor is the lyrics-transcription half of the HeartMuLa pack - a Whisper-based ASR model that was specifically retrained to pick vocals out of music, where normal Whisper tends to melt into the beat. Feed it an AUDIO input and you get back a plain text lyrics string.

Why would you want this in a graph instead of a separate tool? Because the whole pack is a closed loop: transcribe a track with this node, hand the words plus a few style tags to HeartMuLaGenerator, and you get a new song built from those lyrics. It's a SUNO-style "make music from text" pipeline, and this node is the captioning step that lets you recycle existing songs instead of writing lyrics by hand. It also works standalone as a much better-than-Whisper lyrics transcriber for mixing down, covering, or just finally knowing what that chorus says.

How it works

The sibling HeartTranscriptorLoader node loads the model. It scans ComfyUI/models/checkpoints/ for directories, then builds a HuggingFace transformers AutomaticSpeechRecognitionPipeline from a Whisper checkpoint, at float16 by default on auto device (cuda → mps → cpu). Loading is cached, so re-runs don't reload the weights.

HeartTranscriptor takes that pipeline plus an AUDIO tensor, writes it to a temp WAV file, and runs the ASR. For long audio it chunks into 30-second windows with timestamps (enable_chunking on by default). The decode settings - no_speech_threshold, compression_ratio_threshold, logprob_threshold - are the same segment-filtering knobs Whisper's own API exposes, used to drop instrumental-only passages and low-confidence garbage. Output is the lyrics STRING, and since this is an output node, wire it into ShowText/SaveText or straight into the music generator.

The inputs that actually matter

Ignore most of the decode thresholds - the defaults are sensible. A beginner touches four things:

• audio - the AUDIO input. From a LoadAudio node, or from a TTS/source-separation step if you chain one in.
• language - auto by default, or force zh, en, ja, ko, es, fr, de, ru.
• task - transcribe keeps the original language; translate renders everything into English. Translation is handy, but it's a rough pass, not a polished translation.
• enable_chunking - keep it on for songs; flip it off if you hit errors on Apple Silicon (more below).

temperature builds a fallback sequence internally (0.0 → your value), which is how Whisper retries low-confidence words - set it above 0 only if you're getting repeated hallucinations. seed makes runs reproducible.

Installation

Via ComfyUI Manager, search comfyui-kaola-heartmula. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/kana112233/ComfyUI-kaola-heartMuLa.git
cd ComfyUI-kaola-heartMuLa
pip install -r requirements.txt

Real dependencies are slim: torch, torchaudio, transformers>=4.40.0, soundfile. No separate whisper package - it uses the Whisper implementation baked into transformers.

The model itself is a separate download and nothing auto-fetches it:

cd ComfyUI/models/checkpoints
hf download HeartMuLa/HeartTranscriptor-oss --local-dir ./HeartTranscriptor-oss

(The hf command is HuggingFace Hub's CLI; --local-dir needs a recent huggingface_hub. If hf download isn't available, pip install -U huggingface_hub.) The model directory name is what appears in the loader's dropdown. VRAM is friendly - about 1.5GB in float16.

Common issues

• Loader shows "(none found)" - the model directory must live directly under ComfyUI/models/checkpoints/, and it must be the folder you downloaded, not a zip.
• Errors on Mac MPS - the tooltip on enable_chunking says it all: chunked processing can break on Apple Silicon. Turn it off and let the whole track go through at once.
• Lyrics come out garbled on busy mixes - this model is built for music, but it's not magic. The README's own tip: run a vocal separator like Demucs first and transcribe the isolated vocal track.
• OOM on long songs - if chunking is off, a full track has to be held in memory; either re-enable chunking or trim the audio before the node.

One honest caveat: HeartMuLa is a young project, and the ComfyUI wrapper is still basic. The transcription quality is genuinely good, but the surrounding ecosystem (model auto-download, more languages) is thinner than a mature pack. Fine trade if you want local lyrics ASR that speaks music.