Gemini CLIPSeg

Let's kill the confusion in the first line: this node does not call Gemini and needs no API key. It carries the "Gemini" prefix only because it ships in the ComfyUI-OllamaGemini pack. Underneath, it's plain CLIPSeg - a local, text-prompted segmenter. You give it an image and a word ("hair", "jacket", "background") and it returns a mask of that thing. It's effectively the same node as the pack's CLIPSeg registration, just exposed under a second name.

What it does and why you'd use it

Masking is the unglamorous setup for most selective editing in ComfyUI: inpaint only the face, recolor only the shirt, cut the subject out for compositing. All of it needs a mask marking "this region, not that one." CLIPSeg builds that mask from text, so you skip the manual clicking and drawing. That's its whole appeal - one node, one text box, a usable selection in seconds. It belongs to the text-prompted masking family; compared to the SAM-plus-detector route it's lighter and looser, which makes it a fine beginner choice for "roughly grab this so I can edit it" and a poor one for pixel-perfect edges.

How it works

It leans on CLIP's joint understanding of images and language, scoring how well each region matches your prompt and turning those scores into a grayscale heatmap - bright where your thing probably is, dark elsewhere. Because it's a similarity map rather than a hard detector, raw edges are fuzzy, which is why the node hands you cleanup knobs.

The inputs and outputs that matter

• image - the picture to segment.
• text - the single concept to find. A concrete noun beats a sentence.
• threshold (0.4) - where the fuzzy map becomes a solid selection. The main dial: too high shrinks the mask to nothing, too low grabs everything.
• blur (7) - feathers the edge for soft blends.
• dilation_factor (4) - expands the mask a few pixels so an inpaint covers slightly past the object edge and doesn't leave a seam.

Outputs: mask (soft/feathered) and hard_mask (a cleaner binary cut). Feed the soft one into blended compositing or inpainting, the hard one when you want a decisive edge - into an inpaint node's mask input, a Set Latent Noise Mask, or a compositor.

Installing it

ComfyUI Manager: search ComfyUI-OllamaGemini, install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/al-swaiti/ComfyUI-OllamaGemini
pip install -r requirements.txt

then restart. The CLIPSeg weights are small and download automatically the first time you run the node, so that first run needs a connection and pauses briefly to fetch them.

Where people get burned

The name is the first trap - people expect Google's Gemini to be doing clever segmentation and it isn't; this is offline CLIPSeg, and the two behave identically whichever registration you pick, so don't hunt for a difference that isn't there. The second is a vague mask. On busy scenes or fine detail (hair strands, mesh, foliage) the similarity map smears; tighten threshold, simplify the text to a plainer noun, and prefer hard_mask. If you truly need clean hair or semi-transparent edges, CLIPSeg is the wrong tool - a dedicated matting model gives fractional-alpha edges this can't. A blank output almost always means threshold is set above anything the model matched; lower it and try a simpler word.