EmbeddingGemma 2 Encode Audio

Thirty seconds. That's the whole story of this node, and it's better to know it before you build a workflow around it than after.

EmbeddingGemma 2's audio tower reads at most 30 seconds of a clip - the official processing truncates at 480,000 samples, which is 30 s at the model's 16 kHz, which is 750 audio tokens. Feed it a four-minute track and you get the embedding of its first four minutes' worth of nothing; the author checked this directly, and a 60-second clip produces exactly the embedding of its first 30 seconds. Bit for bit, silently, no warning. There's no truncation flag to turn off. If you want the whole song considered, you cut it into chunks, embed each, and combine or compare the pieces yourself.

Not a pack limitation - the model card talks about minutes; the shipped processing caps it. Exactly the kind of gap that costs you an afternoon.

What it does with your waveform

ComfyUI gives you AUDIO as a {waveform, sample_rate} dict, waveform shaped [B, C, N]. This node:

• averages channels down to mono (so a stereo mix collapses) - one embedding per batch item, so a batch of 3 clips gives 3 vectors, labelled audio 0, audio 1, audio 2;
• resamples anything that isn't 16 kHz with scipy.signal.resample_poly, a polyphase filter, because the model's feature extractor only accepts 16 kHz and the official usage leaves resampling to the caller;
• hands each clip over as float32 numpy.

That resample step is why scipy is in the requirements - a real dependency, not a leftover.

Labels are positional. In a Similarity run your best_label will read audio 2, not the filename, so keep your own mental map of batch order.

Inputs

audio - a ComfyUI AUDIO input. Empty waveforms and wrong shapes raise with the offending shape in the message.

dimensions - 768 default, down to 512 / 256 / 128 via Matryoshka truncation with renormalisation. The model card's spacing here is honest: 256 is close to lossless, 128 is noticeably worse for audio, video and images. Since audio here is already a 30-second summary, I wouldn't push past 256.

model - from the loader, with audio encoders enabled. all or text + audio. If you loaded text + image/video, this node errors out saying the loaded modalities don't include audio. That's the only failure mode that isn't your own doing.

Outputs: embeddings (EG2_EMBEDDINGS, wire into Similarity or Rank Images - though Rank Images is image-specific, so in practice Similarity) and report, a JSON string with count, dimensions, elapsed seconds and the model's dtype/device.

What this is actually good for

Two honest use cases, and neither is "understand my music".

Sorting or matching short clips. A folder of 10-second sound effects, synth stabs, ambience beds, and a text description of what you want - this is exactly the job. Under 30 seconds means no truncation, and the model's audio side handles content-type similarity well enough to be useful. Text-to-audio search is SearchQuery on the text side; audio takes no prefix, same as images.

Labelling generated audio in a pipeline. You're producing variations from an audio model and want to keep the one that matches the brief. Encode the brief with Encode Text, encode the clips here, Similarity picks a winner. Same shape as the image-ranking workflow, and the author's own verification bench covered 4-second clips at both 16 kHz and 44.1 kHz stereo, so ordinary generated audio is well inside the tested range.

What it isn't for: comparing whole songs, spotting a section inside a long track, or anything where "the first 30 seconds" is a lie you'd rather not tell. For long-form you need chunked embeddings plus your own aggregation logic - mean-pool the per-chunk vectors, or score each chunk against the query and keep the max. Neither is packaged as a node. You'd be doing it with the batch behaviour here plus whatever maths nodes you like.

Also of note: Encode Video does not touch a video's soundtrack. If you want the audio of a clip, split it out with Get Video Components and run that through this node.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-EmbeddingGemma2
python -m pip install -r ComfyUI-EmbeddingGemma2/requirements.txt
python ComfyUI-EmbeddingGemma2/tools/download_model.py --models-dir ../models/embeddinggemma

ComfyUI Manager: search EmbeddingGemma 2 for ComfyUI. Then restart. The model is about 1.5 GB, downloaded at a pinned revision with SHA-256 verification, into models/embeddinggemma/embeddinggemma-2/.

Two install things that catch people. The pack uses ComfyUI's V3 node API, so an older ComfyUI registers nothing and looks broken. And sentence-transformers 6.1.0 pins huggingface-hub<2.0, moving a fresh environment's hub from 2.2.0 back to 1.33.0 - fine in testing, worth remembering if another pack starts complaining.

Speed: about 0.1 s for a 4-second clip on an RTX 5090 in bfloat16 with everything resident.