CLAP Audio Analyzer

The main Audio Mood Analyzer node measures what a song does - tempo, brightness, energy - and lets a language model translate those numbers into feeling. This node listens differently. CLAP Audio Analyzer computes a semantic audio embedding and ranks it against a list of mood phrases you provide, then hands you the three that match best. It's the difference between "this is 92 BPM with a 2400 Hz centroid" and "this feels like nocturnal fear, submerged sadness, slow emotional collapse."

CLAP is the contrastive audio-language model from LAION (the laion/clap-htsat-unfused checkpoint), the same family of ideas as CLIP but trained on audio–text pairs. Audio goes in one side, a set of text anchors goes in the other, and the node scores each anchor by cosine similarity against the audio embedding. The result is a short vocabulary of affect - not an analysis, a mood read.

How it works

The node takes your AUDIO, resamples it to whatever the CLAP processor wants, embeds it, embeds your anchor phrases the same way, and dot-products the two. The top three matches become semantic_summary, formatted as CLAP: anchor1, anchor2, anchor3. That string is designed to be wired straight into the base node's custom_context input, which is exactly how the pack intends it: the CLAP reading acts as "semantic pressure" on the LLM's mood interpretation without overriding the acoustic analysis. The librosa numbers describe the physics; the CLAP anchors hand the LLM a named emotional vocabulary. Both reach the model in a single analysis call.

The full clap_json output gives you the model name, embedding dimension, the embedding's norm, the ranked matches with scores, and the top-3 inference - useful if you want to see how confident the match was before you trust it.

Inputs that matter

• audio - any ComfyUI AUDIO source.
• clap_text_anchors - one anchor phrase per line. The default set of 15 (dark atmospheric tension, melancholic isolation, nocturnal fear, explosive catharsis, etc.) is genuinely well-chosen for emotional imagery, and you should absolutely edit it. The anchor list is the instrument; it's how you steer what the model listens for. Want brighter moods? Rewrite the list.
• clap_model - HuggingFace model ID, default laion/clap-htsat-unfused. You probably never need to touch this.
• clap_device - auto, cpu, cuda, or mps. auto picks the best available. First run downloads the CLAP checkpoint from HuggingFace, so budget for that (a few hundred MB) and a few minutes of waiting.

The gotchas, and they're real ones

The model is loaded once per ComfyUI session and cached by (model_name, device). Changing clap_device mid-session has no effect until you restart ComfyUI - that's in the source, not a rumor. If you flip it and nothing changes, restart before you blame the node.

The other designed-in behavior: on any error, semantic_summary returns an empty string and the workflow continues. The pipeline treats CLAP as optional by design. If you're running without a GPU (or with a GPU too small for the checkpoint), CLAP will fall back to librosa-only analysis and you won't lose the run - you'll just lose the semantic layer. An error appears in the clap_json output with a "fallback": "librosa_only" marker so you can see it happening.

Install and wiring

Same install as the rest of the pack - ComfyUI Manager, search AudioMoodAnalyzer, or:

cd ComfyUI/custom_nodes
git clone https://github.com/andrea-spoldi/ComfyUI-AudioMoodAnalyzer.git
pip install -r ComfyUI-AudioMoodAnalyzer/requirements.txt

Restart, and you'll find it under audio/analysis. The transformers>=4.35.0 requirement in the pack's requirements.txt is here for this node - librosa alone doesn't need it.

This is a niche node in a niche pack. Reddit barely registers CLAP-based audio analysis in ComfyUI at all, which tells you two things: there's no established orthodoxy for it, and nobody's holding your hand. The design here is actually the sensible version of the idea - semantic embedding as input to a prompt-writing LLM, rather than pretending you can retrieve images directly from an audio embedding. Wire semantic_summary into AudioMoodAnalyzer.custom_context, run the base node, and let the LLM decide what the CLAP reading means visually. That's the intended workflow, and it's the one worth copying.