ForgeAI Music Generator (HeartMuLa)

The first thing to know: this node turns text into actual music - a full song with vocals, not a loop or a jingle. Give it lyrics plus a comma-separated tag string like pop, emotional, synth, female voice, and it autoregressively writes a whole track, then drops a WAV or MP3 into your output folder. It's the ComfyUI wrapper around HeartMuLa-oss-3B, the 3-billion-parameter open music model the HeartMuLa team released under Apache-2.0 in early 2026. No API, no key, no subscription - it all runs on your GPU.

It's also one of the few nodes that makes this workable on consumer hardware. The author (PavonicAI / "ForgeAI") built it specifically to get the model under the 16 GB VRAM ceiling using 4-bit NF4 quantization, tested on an RTX 5070 Ti. Generation sits around 13 GB VRAM; encoding drops to about 8 GB.

Now, expectations. This is not Suno. The model is small and the community consensus is "decent" - think serviceable pop and electronic demos, vocals that mostly track the lyrics, and the occasional MIDI-ish patch sound. Where it genuinely earns its place is iteration inside a graph: you can batch genre tags, sweep CFG, and wire the output straight into your existing audio pipeline instead of tabbing out to a paid web service.

How it works

HeartMuLa is a language model over audio tokens. The node wraps your tags in <tag>...</tag> markers, appends the lyrics, then generates tokens frame by frame - each frame is 80 ms of audio spread across eight parallel codec tracks (HeartCodec's residual quantizers). CFG runs two parallel streams with classifier-free guidance, and temperature plus top_k do the token sampling. When the model emits an audio end-of-song token it stops early, then detokenizes the whole thing to a 48 kHz waveform. A pipeline cache keeps the model loaded between runs, so the second generation is much faster than the first.

The inputs that matter

• lyrics - the vocal content. Structure tags make a real difference: [verse], [chorus], [bridge], [outro] tell it where the sections go. Multi-language lyrics (English, Spanish, French, Japanese, and more) work.
• tags - comma-separated style string, genre first: genre:pop, emotional, synth, female voice. Genre is highest priority; timbre, mood, and instrument come after.
• cfg_scale - how strictly it follows your tags. 2.0 is the sweet spot for clean vocals; 3.0 for rock and uptempo; 4.0+ turns aggressive and can artifact.

The rest are familiar knobs: duration_seconds (5–240), temperature, top_k, fade_out (smoothly ends the track over the last N seconds so it doesn't cut off), output_format (wav or 320 kbps mp3), and quantize (4bit / 8bit / none).

Outputs

audio (ComfyUI AUDIO) wires into other audio nodes; file_path (STRING) is the saved file on disk. The node is an output node, so ComfyUI shows a playable audio preview in its assets panel.

Installation

ComfyUI Manager: search ForgeAI-HeartMuLa, or do it by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/PavonicAI/ForgeAI-HeartMuLa.git
pip install -r ForgeAI-HeartMuLa/requirements.txt

Then restart ComfyUI. The model downloads are the real install: ComfyUI/models/HeartMuLa/ needs the model folder (the recommended pre-quantized 4-bit checkpoint is ~4.9 GB from PavonicAI/HeartMuLa-3B-4bit), plus HeartCodec and tokenizer.json / gen_config.json from the original repo. Dependencies include heartlib (HeartMuLa's inference library), bitsandbytes for quantization, transformers, and torchtune.

Gotchas

• First run downloads a lot. Set aside a few GB and let it finish before judging anything.
• CFG 1.0 crashes - the node auto-clamps it to 1.1, so it's not your problem anymore, just don't be confused by the "clamped" log line.
• MP3 needs PyAV (bundled with ComfyUI). If encoding fails it quietly saves a WAV instead, so check the output extension.
• The model dropdown only lists folders containing a config.json inside models/HeartMuLa/ - keep the structure from the README and you'll see exactly one entry.

If you're coming from the other ComfyUI HeartMuLa wrapper (CRT-HeartMuLa), the big difference here is that models aren't auto-downloaded - you fetch them once yourself, and you get the baked-in compatibility fixes for transformers 5.x and newer torchtune in return.