Replicate mattsays/sam3-image

Segment Anything (SAM) turned "cut this thing out of an image" from a hand-masking chore into a text prompt. SAM3 is the newest generation of that idea from Meta, and this node is mattsays' build of it running on Replicate. The pitch is simple: give it an image and say "person" (or "the red car", "the dog in the corner"), and it returns a mask isolating what you described. If you've been using Impact Pack's SAM integration locally, this is the same concept for a ComfyUI install with no GPU to run it on.

Mechanically it's a Replicate schema node: IMAGE in (tensor → base64), mask out (an IMAGE tensor - a mask you can wire straight into a Mask composite or an inpaint workflow), plus API_JSON. The inputs that matter:

• prompt (default "person") - the text that tells SAM3 what to grab. This is the whole trick of the model: it's not drawing a box, it's finding what your words describe.
• threshold (default 0.5) - confidence cutoff for what counts as the object. Low it if the model keeps missing the thing; raise it if the mask is too greedy and bleeds into the background.
• mask_only (default false) - return just the mask, or the overlaid result. Turn it on if you only need the mask for downstream compositing.
• save_overlay and return_zip - output-side conveniences; save_overlay writes the visualized result, return_zip (default on) bundles multi-object outputs into a zip. If you're getting unexpected zips, this toggle is why.
• mask_opacity (default 0.5) and mask_color (default "green") - just how the overlay preview looks. Purely cosmetic.
• dry_run / force_rerun - pack-wide: test free, and force a fresh call past ComfyUI's cache.

Where this shines in a workflow: the KB's background-removal essay notes segmentation is a different job from matting - a segmentation mask labels each pixel foreground/background, which is structurally wrong for hair, smoke, or glass. SAM3's prompt-following is the counterweight: instead of a global "remove background" it can chase a specific thing in the frame. So it's great for "cut the subject out, keep the interesting background," or for making an inpaint mask for a single object. It's less great for a clean transparent PNG of wispy hair - that's a matting model's job (BiRefNet territory), not a segmentation model's.

Install is the pack standard - ComfyUI Manager (search ComfyUI-API-DockerCPU), or:

cd ComfyUI/custom_nodes
git clone https://github.com/trustypangolin/ComfyUI-API-DockerCPU
cd ComfyUI-API-DockerCPU
pip install -r requirements.txt

Set REPLICATE_API_TOKEN and restart; it's under 🎨 DockerCPU API/🎨 Replicate.

Real-world notes: prompt phrasing matters more than people expect - "person" works, but "the man in the foreground" is the kind of specificity that decides whether you get a clean subject mask or a mask full of the background too. And it's pay-per-call, so when you're tuning threshold and prompt for a batch, dry-run with a single image first and check API_JSON before spending on the whole set. The output mask is an IMAGE tensor, not a ComfyUI MASK tensor - if a downstream node demands MASK, you may need a quick conversion step. That's a small friction point, but worth knowing before you wire it into an inpaint pipeline.