YUE_SM_Model

What this actually is

If you came from image generation, you'll assume the MODEL output means a UNet you hand to a KSampler. Not here. YUE_SM_Model loads the entire YuE2 pipeline - a 3B music language model that turns lyrics into codec tokens - and exposes it on a MODEL socket so the graph stays tidy. The only thing that consumes it is YUE_SM_Sampler. KSampler will stare at it blankly.

YuE is the open lyrics2song model from m-a-p (multimodal-art-projection), and YuE2 is the rewrite: 3B params instead of the original 7B, and it plans the song's musical content as text in ABC notation before it generates any audio. When YuE first landed in January 2025, r/StableDiffusion responded with "Suno AI at home" and 787 upvotes - along with the catch that full songs wanted something like 80GB. YuE2's own release card targets a 24GB NVIDIA card with bf16. The community's frugal route (DeepBeepMeep's ~10GB port) exists if you're stuck below that, but the honest floor for this node is a solid consumer GPU and patience.

Worth knowing who wrapped it: smthemex is the porter behind ComfyUI_EchoMimic and a stack of other heavy research models, and the one thing the community says about his packs is that they run on modest VRAM where the reference implementation doesn't. That reputation is the reason this pack is worth trying at all.

How it works

Two halves get glued together. The folder ComfyUI_YuE/YuE2-3B/ inside the pack ships the architecture code, config and tokenizer (qwen.tiktoken) - but no safetensors. Your downloaded model.safetensors from m-a-p/YuE2-3B is the checkpoint. The node calls the pipeline's from_pretrained with the bundled folder as the source of code and your file as yue_ckpt, then drops the resulting pipeline object on a MODEL output.

One input, diffusion_models: a dropdown listing everything in ComfyUI/models/diffusion_models, with none prepended. Pick your YuE2 file. Treat none as a placeholder - the bundled folder has no weights of its own to fall back on, so there is nothing local for it to load.

Output is a single model (MODEL) socket that goes straight into YUE_SM_Sampler.

Installing it

Grab the pack via ComfyUI Manager (search ComfyUI_YuE) or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_YuE
pip install -r requirements.txt

Then the weights: download model.safetensors from m-a-p/YuE2-3B to ComfyUI/models/diffusion_models/yue2_model.safetensors. The README says "rename from model.safetensor or not" - the folder scan only cares that it's a .safetensors sitting in that directory, so call it whatever you'll recognise in a dropdown.

The pack is written against the new comfy_api.latest node API (io.Schema, a ComfyExtension entrypoint), so an old ComfyUI build won't load it at all. Update before you file a bug.

Where people get burned

• This loader clears ComfyUI's model cache every time it runs. Its helper walks the loaded models, calls unpatch_model on each, empties the cache and prints the peak GPU allocation. That's why YuE doesn't peacefully co-exist with your image model - do songs in a fresh session rather than leaving Flux resident and hoping.
• requirements.txt is short and innocent. It lists torch, omegaconf, torchaudio, einops, numpy, transformers, sentencepiece, tqdm, mir_eval and pretty_midi. But upstream's YuE2 runtime pins its own transformers/torch/numpy, and SheetSage2 (only needed for covers) pins a different set again, with upstream's own documentation telling you to keep them in separate virtualenvs. Inside one ComfyUI you get one environment. If the pack imports fine but dies on a transformers symbol the moment you generate, that pin collision is the reason - not the model file.
• Don't judge the file by its size class. A 3B LM plus semantic and acoustic stages is more than a 3B image model in working memory, and the load happens on the first sampler run. Budget RAM as well as VRAM.