AudioLDM2

Yes, this pack has an audio node. gokayfem's VLM Nodes is mostly about vision-language models, but tucked in the Audio category is AudioLDM2Node, a real text-to-audio diffusion pipeline (AudioLDM-2 from the paper AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Audio Latent Diffusion). You type a description, you get a waveform back, no separate tool, no API. It's also the back half of the pack's "Image to Music" party trick: run an image through a VLM to get a description, then feed that description here.

What it's for

Ambient pads, sound effects, a bed of music under a video - that's the practical envelope. It's not a songwriting tool; it's "generate audio from a text description" in the same way the image side generates from prompts. If you build video or animation workflows in ComfyUI and want a quick soundscape without leaving the graph, this is the node.

How it works

The node pulls the cvssp/audioldm2 weights from Hugging Face into models/LLavacheckpoints/files_for_audioldm2 on first run, then drives a diffusers.AudioLDM2Pipeline. On CUDA it enables model CPU offload by default so it doesn't eat your whole VRAM alongside the diffusion model you're already running. It needs diffusers, torchaudio/soundfile - the pack's requirements bring those in.

The inputs that matter

• text - what you want to hear. "soft ambient pad, slow chord progression" is the kind of thing that works.
• negative_prompt - do use it; "harsh noise, distortion, drums" keeps things clean.
• duration - 1–60 seconds, default 10. Longer takes longer and gets loopier.
• guidance_scale - default 3.5; nudge up for more prompt adherence, down for more variety.
• n_candidates - AudioLDM-2's trick: generate this many candidates and keep the best by CLAP score. More candidates, better pick, slower.
• seed - standard. Same seed, same track.
• extension - wav/mp3/flac for how the file is written.

Outputs and saving

Two outputs: wave_form (a raw audio tensor) and sample_rate (an INT). The node is an output node, so it's meant to be a terminal - wire both into the pack's SaveAudioNode, which writes to ComfyUI's output folder at the extension you picked. You can also hang an oscilloscope-style viewer on the waveform if you have one.

Install

Same as the rest of the pack: ComfyUI Manager (search VLM_nodes) or clone it and pip the requirements:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
python -m pip install -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements.txt

Common issues

The classic one is the model download stalling on first run - it's several hundred MB, give it a minute. If you get an import error about soundfile or diffusers, the pip step didn't complete; run it with ComfyUI's Python and restart. And don't expect a finished song: you'll get a loopable texture more often than a track with structure. That's the model, not the node.