Nodes/Music Production Toolkit/Generate song · MiniMax / YuE2
ComfyUI Node

Generate song · MiniMax / YuE2

The One Node That Expands Into a Whole Music Pipeline

By jplenio·Created about a month ago·Updated 16 days ago· 72
Generate song · MiniMax / YuE2
    • audio
    • sampler_name
    • scheduler
    • generation_json
    ◄profile_json—►
    ◄settings_json—►
    ◄style—►
    ◄lyrics—►
    ◄yue2_checkpointyue2_3b_bf16.safetensors►
    ◄minimax_modelminimax_music3_dit_fp16.safetensors►
    ◄minimax_encoderminimax_music3_text_encoder_pruned_int8_convrot.safetensors►
    ◄minimax_vaeminimax_music3_dav.safetensors►
    ◄tiled_decodetrue►
    ◄cover_source_json—►
    ◄cover_abc—►

    Everything above this node in the workflow is preparation. MusicGeneration is where audio appears - and it does that without sampling anything itself. It's a subgraph emitter: hand it a model profile, a settings record, a Style and a Lyrics field, and it builds the handful of native ComfyUI nodes that load the weights and run the sampler.

    Two things to know before you touch it. It needs a ComfyUI build new enough to expose the native nodes underneath - YuE2GenerateABC, YuE2GenerateMusic and EmptyYuE2LatentAudio for YuE2, the MiniMaxMusic3* pair, KSamplerWithConfig and the safe audio decoder for MiniMax. And it only ever expands the engine you selected. The other engine's nodes are never built, so a MiniMax run never touches your YuE2 checkpoint.

    What you wire in

    profile_json and settings_json come from the two nodes upstream and must agree, or you get a blunt "Song model and generation settings disagree" error. style and lyrics come from the parser (for YuE2, the parser's "caption" socket carries Style). Then the model filenames, which are plain local names, not URLs:

    • yue2_checkpoint - default yue2_3b_bf16.safetensors, resolved from ComfyUI's checkpoints category
    • minimax_model, minimax_encoder, minimax_vae - the DiT, the pruned int8 text encoder and the DAV VAE, from diffusion_models, text_encoders and vae
    • tiled_decode - on by default, useful when a full-track decode is the thing that runs you out of memory

    For covers, also wire the two optional inputs: cover_source_json from the source node and cover_abc from the transcription node. Outputs are audio (the generated track, into your processing chain), sampler_name and scheduler (what actually ran, after any host-side fallback), and generation_json, the record that feeds the production JSON.

    How it works

    For YuE2 it builds the native path: load the checkpoint, plan an ABC score with YuE2GenerateABC (skipped entirely in cover mode - your transcription takes its place, unchanged, by design), pass Style, Lyrics and the score to YuE2GenerateMusic, size the latent from the model's own returned duration, then sample with the settings group the profile selected. MiniMax is a different shape: separate UNET/CLIP/VAE loaders, MiniMaxMusic3TextEncode doing the caption-plus-lyrics heavy lifting with its own text cfg and top-k, a zeroed negative, then sampling.

    One validation is worth knowing: when a Length has been requested, the YuE2 Style must begin with the matching duration line - the node refusing to generate a five-minute arrangement after the two halves of the prompt drifted apart.

    Expectations, honestly

    This is a young, heavy corner of a young, heavy category - the audio layer is the last thing bolted onto ComfyUI, and it's still node packs fighting transformers and torch versions. The first YuE release earned a rough local reputation: a real thread from February 2026 has someone with 32 GB of VRAM finally getting runs out of YuE in ComfyUI and producing "90 seconds of silence," with the top reply pointing them at ACE-Step instead - "leagues better... runs natively in comfy." That's the baseline YuE2 has to beat.

    Its developers report competitive September 2026 evaluation results - treat that as their claim. What's certain: a full song through a 3B model plus a score planner is minutes of GPU time, not seconds, and community signal on YuE2 is close to nonexistent. You're an early adopter here.

    When it breaks

    • Non-finite latents. With the official fp16 MiniMax DiT, some seed/prompt/length combinations write non-finite values for the whole track. It's seed-dependent and no node can fix it - ComfyUI picks the dtype at checkpoint load. --bf16-unet is the cheap experiment; --fp32-unet is the documented one (and doubles VRAM, so switch the artwork branch off if you're on 16 GB). Mostly, just queue again with a new seed.
    • Incoherent-but-finite audio. A CUDA graph / allocation-compiler interaction with the MiniMax text encoder. MINIMAX_MUSIC3_RUNTIME_SAFETY=auto disables capture only on backends flagged as risky, =on forces it everywhere.
    • Short writes on export. If saving dies with a blank AssertionError from SoundFile, that's invalid samples, not a disk problem. The pack validates samples in both savers and retries decoding once with smaller tiles, but it won't replace garbage with silence.
    cd ComfyUI/custom_nodes
    git clone https://github.com/jplenio/ComfyUI-MiniMax-Music-Production-Toolkit.git
    cd ComfyUI-MiniMax-Music-Production-Toolkit
    python -m pip install -r requirements.txt   # same Python as ComfyUI
    

    The Python requirements are light - numpy, scipy, soundfile, imageio-ffmpeg, mutagen, Pillow. The weight is the models: MiniMax Music 3's DiT, text encoder and VAE, or the YuE2 3B BF16 checkpoint, plus FLUX.2 Klein and FlashSR if you leave those stages on. The model-check node reports exactly what's missing before you pull anything.

    CategoryMusic Production Toolkit/generation

    Inputs (11)

    NameTypeDefaultDescription
    profile_jsonSTRINGJSON profile of the selected song model (id, display name, duration window, prompt hard limit, capabilities). Everything downstream adapts to it.
    settings_jsonSTRINGResolved model-settings JSON from the music settings node: active sampler group, clamped duration, seed handling and the instrumental-check options.
    styleSTRINGStyle or caption text that drives the song model. For a cover this is the style the studio produced.
    lyricsSTRINGFinal MiniMax Music Lyrics/structure field. For pure instrumentals this should contain only supported structural tags; for vocal tracks it contains tags plus singable lyrics.
    yue2_checkpointSTRINGyue2_3b_bf16.safetensorsYuE2 checkpoint file. Only read when the active profile is YuE2 or YuE2 Cover.
    minimax_modelSTRINGminimax_music3_dit_fp16.safetensorsMiniMax Music 3 diffusion model. Only read when the active profile is MiniMax Music 3.
    minimax_encoderSTRINGminimax_music3_text_encoder_pruned_int8_convrot.safetensorsMiniMax Music 3 text encoder. Only read when the active profile is MiniMax Music 3.
    minimax_vaeSTRINGminimax_music3_dav.safetensorsMiniMax Music 3 VAE. Only read when the active profile is MiniMax Music 3.
    tiled_decodeBOOLEANtrueDecode long audio in tiles instead of all at once. Slightly slower and much lighter on peak memory; invalid output retries once with smaller tiles.
    cover_source_jsonoptSTRINGSource audio identity and shared transcription mode from Cover song / Source audio. Used only for YuE2 Cover; the filename owns the final title.
    cover_abcoptSTRINGOriginal SheetSage2 transcription. YuE2 Cover passes this score unchanged to music generation and uses it to guide the LLM arrangement.

    Outputs (4)

    NameTypeDescription
    audioAUDIO—
    sampler_nameSTRING—
    schedulerSTRING—
    generation_jsonSTRING—