SD HeartMuLa Model Loader

Every music workflow in this pack starts here, so this is also the node where the surprise bill arrives. The first time you hit Run, SD HeartMuLa Model Loader quietly downloads three things from HuggingFace - the ~6GB HeartMuLa-oss-3B transformer, the HeartCodec-oss audio codec, and the shared tokenizer.json / gen_config.json - into ComfyUI/models/HeartMuLa/. Plan for a real download before your first song, not a quick refresh. It caches after that, and you'll see Using cached model in the console on reruns.

Under the hood it's a thin wrapper around the official HeartLib HeartMuLaGenPipeline, the open-source (Apache 2.0 after an initial CC-BY-NC stumble) multilingual music model family that shipped in January 2026. The loader resolves your device (CUDA, then Apple Silicon MPS, then CPU), picks a dtype, and hands back a pipeline dict that the Conditioning, Sampler, and Decode nodes all share. One pipeline is loaded per cache key, and it enforces a one-model policy - loading a new config unloads the old one to free VRAM.

The inputs that matter:

• model_version - 3B is the only real choice. The loader will raise a cheerful "7B Model Coming Soon!" error if you pick 7B, because that checkpoint genuinely hasn't been released. Any folder with a config.json inside models/HeartMuLa/ shows up here too, which is how you point at a fine-tune.
• codec - oss by default; custom codec folders (anything with "Codec" in the name) appear as options.
• memory_mode - auto is genuinely the right default. It reads your free VRAM and picks normal/low/ultra for you. On 8–10GB cards it jumps to ultra, which aggressively empties caches between stages so you actually finish the run.
• precision - auto means fp16 on CUDA/MPS, fp32 on CPU. You can force bf16 on NVIDIA if you want, but there's little reason to.
• use_4bit - 4-bit quantization via bitsandbytes. CUDA only; on MPS or CPU it silently disables itself and falls back to full precision, so don't wonder why a Mac toggle "didn't work."
• force_reload - only for when you've swapped files on disk and want to bypass the cache. Otherwise leave it off.

The one output, model (HEARTMULA_MODEL), fans out to Conditioning, Sampler, and Decode - wire all three.

The honest VRAM picture. The README says ~12GB at fp16, ~6GB at 4-bit, and the memory modes stretch that further. Real-world reports from the HeartMuLa release threads run closer to 19–20GB on a 3090 with the raw library and zero optimization; other Comfy wrappers measured ~12GB at bf16 and ~8GB at 4-bit for a 60-second generation. On a 12GB card, expect auto to put you in low or ultra mode, and accept that generation is slow - this model is not a speed demon. 4-bit is the difference between "fits" and "doesn't," so if you're under 16GB, flip it on.

If the loader throws a FileNotFoundError, it's not broken - it's telling you it's about to download the model and the files simply weren't there. The bigger gotcha is dependency collisions: this pack pins torchao==0.6.1 and wants torchtune, transformers, vector-quantize-pytorch and modelscope, all installed into the same Python environment as everything else you run. The ComfyUI Manager install path handles most of it, but when another node's requirements fight yours, this pack is often the one that loses - check the console for a missing torchtune or torchao import before you assume the loader itself is at fault.