DiffRhythm2歌曲生成

Most music nodes in ComfyUI give you a loop or a texture and call it a day. DiffRhythm2 gives you an actual song - with sung vocals, structure, and lyrics you wrote. It's a ComfyUI wrapper around Xiaomi's Apache-2.0 DiffRhythm2 model, and it's about as close as local open weights get to "Suno, but on your machine." The first DiffRhythm made the rounds on r/comfyui in spring 2025 as the "very fast music generator"; this second version is the one that takes the idea somewhere useful.

What it actually does

You feed it three things: a text description of the style, the lyrics, and how long the song should be. It generates a full track - verses, choruses, a vocalist singing the words - and hands you back an AUDIO tensor you can play, save, or mux onto a video. No API, no key, no Suno subscription. All local.

Under the hood it's a conditional flow-matching model (a DiT backbone, sampled with torchdiffeq's Euler solver at ~20 steps) that produces a mel spectrogram block by block, then a BigVGAN-style decoder turns that into 24 kHz audio. The style isn't invented from thin air: a MuQ-MuLan model embeds either your style prompt or a reference clip, and an XLM-RoBERTa text encoder plus a Chinese/English grapheme-to-phoneme tokenizer turns the lyrics into phonemes the model can sing. It's a heavier stack than your usual SD checkpoint, and the first run downloads four model repos into ComfyUI/models/TTS/DiffRhythm - a few GB, so budget for a slow first generation.

The inputs that matter

Only three are required, and one of them is doing most of the work:

• 音乐风格提示词 (style prompt) - a comma list of tags like Vocal, Pop, Synthesizer, Piano, Electric Guitar, Happy, Romantic. The README's example works well as a starting point. The node refuses to run with neither this nor a reference track, so put something here.
• 歌词 (lyrics) - the actual text, Chinese or English. This one accepts a wire (forceInput), so you can pipe lyrics straight from an LLM or text node if you want a full prompt→song chain.
• 歌曲最大长度 (max length, seconds, 10–500, default 210) - that's ~3.5 minutes. Longer means more blocks to generate and more time, not more VRAM at once, since it samples block by block.

The optional half is small: 参考音乐 (an AUDIO you wire in from a LoadAudio node) trades the style prompt for "make it sound like this clip" - the code grabs a random 10-second window of it. Then 步数 (steps, default 20), cfg (default 2.0 - this model likes low CFG, don't crank it like SD), seed, and 卸载模型 (unload after generate, default on).

Installing it

ComfyUI Manager finds it as ComfyUI_DiffRhythm2, or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/billwuhao/ComfyUI_DiffRhythm2.git
cd ComfyUI_DiffRhythm2
pip install -r requirements.txt
# restart ComfyUI

The one install step everyone skips and then regrets: espeak-ng. The phonemizer dependency needs it for English lyrics. On Linux that's apt-get -qq -y install espeak-ng; on Windows install espeak-ng 1.52 and set the PHONEMIZER_ESPEAK_LIBRARY env var to your libespeak-ng.dll. Miss this and you'll get an import or g2p error the moment you generate.

Where people get burned

• Vocals can garble. This is the honest ceiling of local song generation, not a bug in the pack. Community reports on DiffRhythm2 (and ACE-Step, its main rival) consistently say the same thing: instrumentals are shockingly good, sung words are sometimes mush. Short, clean lines in the structure tags help.
• Use the structure tags. [start] [intro] [verse] [chorus] [outro] [end] (plus [bridge], [solo], [hook], [break], [space]) tell the model where the song actually goes. Without them you get a wall of repeated singing. If you forget [start], it's auto-added.
• Unloading is on by default. With 卸载模型 left on, the model frees VRAM after each song - good if you're chaining into video, but it re-downloads nothing yet re-loads several GB of weights every run. If you're iterating on lyrics, flip it off. And note seed = 0 means "don't seed" - set a nonzero value if you want reproducible takes.

Wire the audio output into any audio playback/save node, or into a video muxer if the song is the soundtrack. It's not Suno yet, and it won't be for a while - but for a private, unlimited, lyrics-faithful song on your own GPU, it's the real thing.