🎵 ACE-Step 1.5XL Audio to Latent

Every ACE-Step workflow that touches existing audio has the same first step: turn a .wav into a latent. Audio to Latent is that step. It takes a loaded AUDIO clip and the ACE 1.5 VAE, encodes the clip into the latent domain, and hands you back the tensor everything downstream expects.

This is the node you reach for when you're not starting from scratch. Extend a track by generating a new outro, repaint a bad middle section, feed a voice sample into a reference node - all of those need your source material encoded first. Load Audio → Audio to Latent → whatever-edit-node is the backbone of every non-text-to-music workflow in this pack, and the example workflows in the repo are built exactly that way.

How it works

Mechanically it's a VAE encode with one piece of polish: it checks sample rates. The ACE 1.5 VAE operates at its own sample rate, and if your source file differs, the node resamples it with torchaudio before encoding - so a 44.1kHz file and a 48kHz file both come out consistent. The latent you get back is the ACE-Step shape, [batch, 64 channels, frames], tagged "type": "audio" so the other ACE-Step nodes know what they're holding.

Two things worth knowing about the encode itself. First, the VAE it uses is the dedicated ace_1.5_vae.safetensors - this isn't an image VAE, and feeding it anything else is how people end up with shape errors. Second, the node is honest about time: it recomputes the actual duration from the encoded frame count, not from anything the source file claimed.

The inputs and outputs

• audio - an AUDIO from a Load Audio node (or anything producing AUDIO, like a video's audio track).
• vae - the ACE 1.5 VAE, loaded via a standard VAE Loader.

Outputs: latent (the encoded clip, wire it into Extend/Repaint/Edit/Reference), plus seconds and latent_frames as a convenience - they're exactly the numbers you need to set the Text Encode node's duration to match the clip you just encoded.

Install

ComfyUI Manager ("ComfyUI-ACEStep") or:

cd ComfyUI/custom_nodes
git clone https://github.com/starsFriday/ComfyUI-ACEStep.git

then restart. No pip dependencies beyond what ComfyUI already ships (torchaudio for the resampling). The models come separately from the ACE-Step 1.5 repo: acestep_v1.5_xl_turbo_bf16.safetensors, a qwen_0.6b_ace15/qwen_4b_ace15 CLIP, and ace_1.5_vae.safetensors. This is the XL tier, so plan for 12GB+ VRAM.

Gotchas

The failure mode you'll actually hit is a shape error complaining about channels. ACE-Step 1.5 latents have exactly 64 channels, and the node validates this - so a latent that came from the wrong VAE (or a latent shaped like a video or image latent) fails loudly instead of silently corrupting your song. That's a feature. The subtler issue is sample-rate handling on very long clips: resampling plus encoding is cheap, but there's no auto-trimming here, so whatever duration your source file has is what you get. If your reference audio has a long silent intro, trim it in an editor first - you'll be glad you did.