EmbeddingGemma 2 Encode Video

You've got a folder of clips from a batch and one line of text describing what you wanted. This node tells you which clip is it. Same idea as image ranking, except the unit is a whole video and the answer is a single vector - no per-frame timelines, no scene splitting.

That "one vector for the whole clip" is the important thing to internalise. It's an average impression of the video, not a search inside it. Ask it "does this clip contain a dog" and you may get a useful answer. Ask it "at what second does the dog appear" and you're using the wrong node.

How the frames get chosen

The node takes ComfyUI's VIDEO input, pulls its components, and hands the processor an array of frames plus the metadata it needs: frame rate, total frame count, duration. With that metadata the official processing does what it does to a video file: 1 frame per second, capped at 32 frames, spread evenly across anything longer than 32 seconds.

So a 5-second clip is 5 frames. A 3-minute clip is still 32 frames, sampled across the whole thing. The embedding of a long clip is therefore the embedding of a sparse storyboard of it - enough for "is this a beach or a city street", not enough for detail.

The soundtrack is ignored. Encode Video reads no audio whatsoever. If the sound matters, take it off with Get Video Components and run it through Encode Audio separately; you'll get two embeddings and can compare them independently. That's also the only way to get audio+video as one thing, and it isn't really one thing - it's two vectors in the same space.

The memory trap (this is the one)

ComfyUI's video loader decodes every frame into memory before the node sees any of it. At 832×480 that's roughly 4.8 MB per frame as float32. A one-minute, 30 fps clip is 1,800 frames, so about 8.6 GB of RAM - and then the node throws away all but 32 of them.

If you're encoding anything longer than a few seconds, trim or downscale first. This is a property of ComfyUI's video handling, not a bug in the pack - the README says so plainly, which is more honesty than most node authors manage - but it will absolutely look like this node's fault when your run dies feeding it a long clip.

Inputs and outputs

video - a ComfyUI VIDEO. It needs frames and a sane frame rate; a video with zero frames or a zero fps gets a clear error instead of a garbage embedding.

dimensions - 768 by default, truncatable to 512 / 256 / 128. Same Matryoshka rule as everywhere in this pack: 512 and 256 are close to lossless, and 128 is where video and images visibly degrade. Your text query has to use the same length or the comparison nodes refuse.

model - from the loaders, with image/video encoders enabled. modalities set to text + audio loads a model with no vision tower, and this node will tell you so.

Outputs: embeddings (labelled video - literally the string, since there's only one item) and a report JSON with count, dimensions, seconds and the usual model details.

One nice consequence of that single label: when you use Similarity to score a text query against your clip, best_label comes back as video and best_score as the number you actually care about. Hence the shipped video_search workflow: five descriptions, one clip, which description wins.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-EmbeddingGemma2
python -m pip install -r ComfyUI-EmbeddingGemma2/requirements.txt
python ComfyUI-EmbeddingGemma2/tools/download_model.py --models-dir ../models/embeddinggemma

Or search EmbeddingGemma 2 for ComfyUI in ComfyUI Manager and restart. About 1.5 GB of model, SHA-256 verified against a pinned revision, into models/embeddinggemma/embeddinggemma-2/.

Two of the shipped workflows use video - rank_frames and video_search, both reading input/example.mp4, so drop any clip there or point Load Video at your own.

Notes from the verification, and one honest limit

Encode Video reads the exact frames ComfyUI decoded - checked against an independent 8-bit decode of the same file, about 1.3/255 average pixel difference, cosine 0.9996 between the embeddings. In practice the frame-source difference doesn't matter.

Speed on the author's RTX 5090: a 5-second video in 0.13 s. The decode is the slow part, not the model.

And the caveat that applies to the whole pack - this was verified on Windows 11 with an RTX 5090 and ComfyUI v0.38.0, plus CPU CI. Linux GPU, macOS and AMD weren't tested with the real model. Nothing in the mechanism is exotic, but if you're on an unfamiliar stack you're the first one through.

The pack is written against ComfyUI's V3 node API, so an older ComfyUI won't register these nodes at all. Update first, debug second.