🧩 Segment Anything

The fastest way to cut a subject out of an image by describing it. Type "dog", pick a SAM checkpoint from a dropdown, plug in your image, and the node hands you a clean mask of the dog without a single brushstroke or bounding box. This is the flagship convenience node of the SDVN Segment Anything pack, and it exists for the person who wants text-prompted segmentation in the fewest possible wires.

What it actually is

This pack is a ComfyUI port of continue-revolution's sd-webui-segment-anything extension for A1111, and the README is explicit that output matches the original for the same input. If you ever used that extension, this is the same two-model dance: GroundingDINO reads your text and proposes boxes, SAM (Segment Anything) refines each box into a pixel-perfect mask. Here the whole dance is wrapped in one node that internally loads both models for you, runs the segmentation, and even builds the preview.

Where it slots into a workflow: out comes a mask, and a mask is the currency of inpainting, background removal, and regional control. Feed it to VAEEncodeForInpaint, composite it with ImageCompositeMasked, or use it to isolate a subject before a detail pass. The KB's background-removal doc makes the honest point that for plain cutouts you'd reach for BiRefNet these days - this node's job is the selective case, where you want the mask of one named thing, not "everything that isn't background."

How it works

GroundingDINO takes your prompt and the image, and returns candidate boxes with confidence scores. Two knobs decide what survives: threshold (default 0.3) filters boxes by objectness, text_threshold (default 0.25) filters by how well the detected phrase matches your words. Anything that survives gets handed to SAM as a box prompt, and SAM - set to single-mask output - returns one tight mask per detection. Then the node zeroes every pixel outside each mask.

The inputs that matter

Only five, and you mostly touch three:

• prompt - the plain-English thing to find: dog, person, the red car. You can list several with dots: dog . cat . person.
• sam_model - one of the seven SAM checkpoints. sam_vit_b (375MB) is the sane default; sam_vit_h (2.56GB) is the quality pick if your GPU has room.
• dino_model - GroundingDINO_SwinT_OGC (694MB) is the standard workhorse; SwinB is bigger and slower.
• threshold / text_threshold - lower them when you're getting nothing, raise them when you're getting junk boxes.

Outputs

You get IMAGE and MASK. The image has everything outside the mask zeroed out - the subject on black, ready for compositing. The MASK is the cutout itself, and it's the one you'll actually wire into inpainting. The node also shows you an alpha preview (subject kept, background transparent) in its UI box, so you can judge the result at a glance.

Install and first run

Through ComfyUI Manager, search for SDVN Segment Anything. Or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/StableDiffusionVN/SDVN_Segment_Anything
cd SDVN_Segment_Anything
pip install -r requirements.txt

then restart ComfyUI. Requirements are light - segment_anything, timm, addict, yapf - but the models are not. The first run quietly downloads your SAM checkpoint, a GroundingDINO config and weights, plus the bert-base-uncased text encoder (~440MB) into ~/.cache/huggingface. That's 1–3.5GB before you see a single mask, and the console goes quiet while it happens, so don't assume it hung. If downloads crawl, the README suggests setting HTTP_PROXY / HTTPS_PROXY before launching. If the bert download is the problem, drop the model files into ComfyUI/models/bert-base-uncased and it'll use those instead.

The honest catches

Because this node loads its models internally every time, each run rebuilds both models. Fine for occasional use. But if your workflow has several segment steps, the modular version - the two loaders feeding SDVN DinoSAMSegment - loads each model once and shares it across nodes. And the classic failure mode: if your prompt matches nothing, you get back an empty (all-black) mask with no error, so when the mask looks wrong, suspect the prompt and thresholds before suspecting the node.