Zero-Shot Audio Classification

Zero-shot classification is the trick where you don't train anything - you just name the categories and the model scores the input against them. This node brings that to audio using CLAP, the audio sibling of CLIP: give it a path to a sound file and a comma-separated list like "speech, music, noise," and it returns a JSON ranking of how well each label fits the clip. Out of the box it answers "is this speech or music?" but since the labels are yours, you can ask things like "bird song vs. traffic" or "electric guitar vs. acoustic" without hunting for a fine-tuned model.

It's from kadirnar/ComfyUI-Transformers, the HF-pipeline wrapper pack. Default model is laion/clap-htsat-unfused (~500 MB), the LAION contrastive language-audio pretraining model that maps audio and text into a shared embedding space.

How it works

Mechanically: pipeline("zero-shot-audio-classification", model=model_name) with the audio path and your candidate_labels (which the node splits on commas and trims). CLAP embeds the audio and each label text, then ranks by similarity. The output results_json is a JSON array of {label, score} pairs, best first - like [{"label": "speech", "score": 0.92}, ...].

Same structural caveat as the pack's video node: the input is a filesystem path, not a ComfyUI audio type. There's no AUDIO socket. You need the sound on disk first - a WAV or MP3 sitting somewhere reachable - and you point the node at it. ComfyUI's audio ecosystem is thin to begin with, so this is very much a "bring your own file" node.

Inputs and outputs

• audio_path - STRING, absolute path to the audio file. Default is empty; feed it a real path.
• candidate_labels - STRING, comma-separated. Default "speech, music, noise". Label wording matters a lot - descriptive phrases like "a person speaking" often score better than terse words.
• model_name - default laion/clap-htsat-unfused; free text.
• Output: results_json - STRING.

The candidate_labels field is the whole trick. Keep labels distinct - "speech" vs. "music" works; "speech" vs. "human speech" confuses the model because they're nearly the same embedding. If scores come out suspiciously flat, the labels are too close together.

Installing it

Pack-standard:

cd ComfyUI/custom_nodes
git clone https://github.com/kadirnar/ComfyUI-Transformers
cd ComfyUI-Transformers
pip install -r requirements.txt

or search "ComfyUI-Transformers" in ComfyUI Manager, restart. The README's cd custom/nodes is a typo. Requirements include soundfile and scipy, which the audio pipeline needs for decoding - the install handles those. First run pulls CLAP from the Hub to ~/.cache/huggingface and that download is several hundred MB.

Where people get burned

• Empty audio_path - default value, immediate error. Point it at a real file first.
• CLAP can be picky about audio. Resampling and format handling live inside the pipeline, but very long or weirdly encoded clips can misbehave. A short, clean WAV is the safe test.
• Model reload on every run. Like the rest of this pack, the nodes bypass the cached loader, so each execution pays a load penalty. Fine for occasional checks, wasteful in a loop.
• Flat scores from overlapping labels. See above - make your categories semantically far apart.

The honest take: this is the most niche node in an already-niche pack, because ComfyUI workflows rarely have audio lying around. But if you're doing any sound-adjacent work - voice-over routing, classifying audio samples before they feed an audio-to-video pipeline - having CLAP as a no-API-key, local, label-your-own-classifier is quietly powerful.