HeartMuLa Music Generator

Here's the thing nobody says about open-source music AI: the vocals are usually the weak link. ACE-Step gets instrumentals right and mumbles through lyrics; Suno does both but lives in the cloud and wants a subscription. HeartMuLa flips the script - a 3B music foundation model whose sung vocals are genuinely good and whose lyric adherence is the best of the open bunch. This node drops it into ComfyUI. No API, no key, no bill: you type lyrics, pick some genre tags, and a full song - sung, with a beat - lands in your output folder as an mp3.

That framing matters. Audio is the newest, thinnest layer in the ComfyUI stack - real and capable, but bolted on in bespoke packs with their own dependency trees. HeartMuLa is the 2026 entrant in the "local Suno" race, and its angle is that it treats lyrics as a first-class input. If you're scoring a video, testing a hook, or just want original music, this is the node you reach for.

How it works

Under the hood this is a packaged HeartMuLaGenPipeline - the heartlib code ships inside the pack's util/ folder, so there's no separate repo to manage. Your lyrics and tags get tokenized (tags wrapped in <tag>...</tag> markers) and fed to the HeartMuLa model, which doesn't output a waveform at all. It generates audio tokens across eight parallel codebooks - that's the "Multi-Attention Latent Alignment" - and a second model, HeartCodec, decodes those tokens back into sound. CFG, top-k and temperature work the way they do on the image side, because it's literally classifier-free guidance applied to audio tokens.

The inputs that matter

• lyrics - multiline text. Structure it like a real song ([Verse], [Chorus]). This is where the track comes from, and the model mostly sings exactly what you write.
• tags - comma-separated style hints, e.g. piano,happy,wedding. The most common complaint is "it ignores my tags," and the fix is usually the next field.
• cfg_scale - default 1.5. This is the dial people get burned on: the model tends to zero out the tags at low values. Community consensus is that 2–3 gives solid results, and users chasing strict genre control go up toward 5.
• version - 3B-happy-new-year (default, a seasonal variant), 3B, 7B. The 7B is a trap: it's listed in the dropdown but the weights weren't public, so picking it means a model-not-found error. Stick with the 3B options.
• max_audio_length_ms - default 240000 (4 minutes), up to 10 minutes. Longer is slower and hungrier on VRAM.
• topk and temperature - the usual sampling knobs; temperature 1.0 is a fine starting point.

Outputs

• audio_output (AUDIO) - wire this to a PreviewAudio node or any audio save node to hear it. The shipped example workflow goes straight to PreviewAudio.
• filepath (STRING) - the full path to the generated mp3 in ComfyUI's output directory.

Installation

ComfyUI Manager can find it (search "HeartMuLa_ComfyUI"), or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/benjiyaya/HeartMuLa_ComfyUI
cd HeartMuLa_ComfyUI
pip install -r requirements.txt

That requirements list is where the hobby ends: torchtune, transformers>=4.45, sentencepiece, accelerate, omegaconf, plus torchaudio, with the README telling you to add soundfile and torchao if imports fail. On Windows you also need a full-shared FFmpeg build on your PATH - the classic Windows audio-node tax.

Then the real lift: download the model weights. The pack looks in ComfyUI/models/HeartMuLa and is picky about folder names - the generation pipeline wants a HeartCodec-oss-20260123 folder and a HeartMuLa-oss-<version> folder matching your selection, plus tokenizer.json and gen_config.json at the base. Get those exact names wrong and you'll get a FileNotFoundError that reads like a corrupt install when it's really a rename. The README's commands:

cd ComfyUI/models
hf download HeartMuLa/HeartMuLaGen --local-dir ./HeartMuLa
hf download HeartMuLa/HeartMuLa-oss-3B --local-dir ./HeartMuLa/HeartMuLa-oss-3B
hf download HeartMuLa/HeartCodec-oss-20260123 --local-dir ./HeartMuLa/HeartCodec-oss-20260123

Common issues

• VRAM. The 3B model in bf16 sits around 16GB allocated; the pack now lazy-loads to squeeze onto 12GB cards, and FP4/NF4 quantization drops it to ~8.3GB at the cost of roughly 15% slower generation. If you're under 12GB, this node is a rough fit.
• Slow generation. The pack's own benchmarks show ~45–50s per 20 seconds of audio on a 5060 Ti; music token decoding is just heavy. The pack caches the loaded pipeline, so the second generation is noticeably faster than the first.
• It ignores your tags. Raise cfg_scale before you blame the model.
• Dependency hell. transformers/torch conflicts are the default failure mode of audio in ComfyUI, and HeartMuLa's stack (torchtune + transformers + bitsandbytes) is right in the blast radius.