DiffSynth: ACE-Step Inference

This is the node that makes music. Text prompt plus lyrics in, an AUDIO out, and at 48kHz if the pipeline's vocoder reports its own rate. ACE-Step 1.5 is the model everyone means when they say "local Suno" - it runs under 4GB with offload, it does LoRA training like an image checkpoint, and it is genuinely good at instrumentals. Vocals are the weak suit, and it's the same complaint people level at Suno: "the sound quality is super clear. Feels better than Suno in parts… The lyrics are absolute garbage."

Own that trade-off before you spend an evening on it. Ambient, backing tracks, game loops, genre pastiche: excellent. A pop song with a lead vocal you actually want to keep: often a coin flip.

What the node exposes

Required: pipe (from the ACE-Step Loader), prompt, lyrics, seed, and num_inference_steps - default 8, because 1.5's turbo checkpoints are few-step. The XL SFT template runs 50. Fifteen seconds of queueing versus a couple of minutes, so try 8 first and only climb if the mix sounds thin.

The optional sockets are where the actual music lives:

• cfg_scale (default 1.0), shift, denoising_strength - guidance and schedule knobs. Leave them alone until you have a baseline you like.
• task_type - free text, default text2music. This is the task the pipeline is handed, not an enum, so mismatch it and you get nonsense.
• duration (default 60 seconds) and bpm (100), keyscale ("B minor"), timesignature ("4"). The tempo/key fields are hints, and they're the difference between "a song" and "a song that fits the video you're scoring". Existing templates run duration at 160 - ACE-Step's ceiling is around four minutes.
• vocal_language - free text, default unknown; set it to the language your lyrics are actually in. The shipped template writes Chinese lyrics and sets zh.
• audio_cover_strength, repainting_ranges, repainting_strength - ACE-Step's cover and repaint features. Repaint wants the region text plus a strength; the cover strength tells it how much of a reference take to keep.
• reference_audios, src_audio, input_audio - the three audio inputs. Wire a Load Audio node: the pack converts the ComfyUI AUDIO (waveform plus sample rate) into what the pipeline expects, so you can hum a melody, cover a track, or repaint a section.
• tiled with tile_size (512) / tile_stride (256) - long-form chunking. The template has tiling on.

Output: audio, one ComfyUI AUDIO. Into Save Audio, or a Preview Audio if you just want to hear it.

Lyrics are structured text, not prose

Write them the way the template does - [Verse 1], [Chorus], [Instrumental Break - Synth Brass Melody], and a line break per phrase. The section tags steer arrangement; without them you get a blurry two-minute strum. An empty lyrics box is the instrumental path, and honestly, that's the mode this model wins at.

Install and the one thing to check

# Manager: search "DiffSynth-ComfyUI" → Install → restart
pip install git+https://github.com/modelscope/DiffSynth-Studio.git
cd ComfyUI/custom_nodes && git clone https://github.com/modelscope/DiffSynth-ComfyUI.git

Easiest correct start: Template tab → any DiffSynth ACE-Step workflow, then edit prompt and lyrics in this node. That hands you the four ModelConfig nodes and the Loader wiring already done.

Troubleshooting

Vocal mush is the model, not your settings - spend the effort on the prompt describing the arrangement, and keep the lyric lines short. If the render starts out as a hiss and resolves late, you're under-stepping for the checkpoint; XL base wants far more than 8. Audio in the wrong tempo range usually means bpm got left at 100 while you prompted for something fast. And if you're on this pack rather than the core ACE-Step nodes at all: you're in a niche - across two years of Reddit I could find roughly 190 threads mentioning DiffSynth and exactly two naming this ComfyUI plugin by name, so almost any workflow you find online is built for a different node set. The pack's own templates are your documentation.