Jags-Clipseg

Want a mask of "person" without training anything or clicking SAM points? Jags-Clipseg does it from a text prompt. You feed it an image, type what you want isolated, and out comes a binary mask plus two previews. It's zero-shot segmentation: the model has never seen your image, and it still finds what you named.

This is the text-grounded masking that keeps classic mask-based inpainting alive. Instruction-edit models are great at whole-image changes, but they can't leave the rest of the frame bit-identical the way a mask can. If your job is "change only the object I name," a mask is still the right tool - and this node hands you that mask from a sentence.

How it works

Under the hood it's CLIPSeg (CIDAS/clipseg-rd64-refined): a frozen CLIP text-and-image encoder with a small segmentation decoder bolted on. The node runs your prompt and image through the CLIPSeg processor, gets raw logits out, then applies a post-processing chain: sigmoid → threshold → Gaussian blur → normalize → dilate. Each step is a knob you can turn.

The pipeline order matters for tuning. The threshold cuts out low-confidence pixels first (a mask that's too generous → raise it), the blur softens the remaining edges, and the dilation grows the mask outward so you don't shave off the object's edge. Defaults are blur 7, threshold 0.4, dilation 4 - sane starting points, but objects with fuzzy boundaries (hair, fur) want the blur up and a lower threshold.

Inputs and outputs

• image (required) - your input.
• text (required) - what to mask. "person", "chair", "the cup on the table". It's a single line, not a paragraph, and it's zero-shot, so plain nouns work best.
• blur (0–15, default 7) - Gaussian blur on the mask, softens edges.
• threshold (0–1, default 0.4) - binarization cutoff. Higher = tighter mask.
• dilation_factor (0–10, default 4) - grows the mask to avoid clipped edges.

Outputs are Mask (MASK), Heatmap Mask and BW Mask (both IMAGE previews overlaid on your input). Wire the Mask output into an inpainting pipeline - Set Latent Noise Mask, or any node that takes a MASK - and use the previews to eyeball whether the mask actually covers what you wanted before you commit.

Setting up the model

This is where people trip. The node looks for CLIPSeg weights in ComfyUI/models/clipseg/. The README says to download the CIDAS/clipseg-rd64-refined repo and drop its files there, config.json included. But here's a nice trick in the code: if that folder doesn't exist at all, the node falls back to pulling CIDAS/clipseg-rd64-refined straight from the HuggingFace hub on first run. So your two reliable paths are:

1. Let it auto-download - don't create the folder, first run fetches the weights.
2. Or download the repo and fully populate models/clipseg/.

The bad path is creating an empty clipseg folder: then from_pretrained points at an empty local dir and fails. Either populate it completely or don't create it.

One real gotcha: transformers is not in the pack's requirements.txt, but this node imports CLIPSegProcessor and CLIPSegForImageSegmentation from it. If you get an import error, that's why:

pip install transformers

Installing

ComfyUI Manager → search Jags_VectorMagic → install → restart, or clone the repo into custom_nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/jags111/ComfyUI_Jags_VectorMagic

Remember the pack installs a heavy requirements list (ultralytics, librosa, clip-interrogator, …) regardless of which node you actually use.

Tuning notes

• Threshold is your main dial. If the mask bleeds into the background, push threshold toward 0.6. If the object comes back half-missing, drop it toward 0.3.
• Text specificity beats text length. "person" is fine; "person with a red hat" usually collapses to just "person". If one prompt can't cover a multi-part subject, mask each part with a separate Jags-Clipseg and merge with Jags-CombineMasks (same pack).
• It's a small model and runs on CPU fine - a second or two per image, not the minutes a diffusion step costs. Experiment freely.

Zero-shot masking by sentence, no training, no SAM install. For "mask this one thing so I can change only it," it's hard to beat.