SAM Segmentor

This node runs SAM - the Segment Anything Model - to carve an image into distinct regions and hand you back a segmentation map. Unlike UniFormer or OneFormer, which label pixels by category (this is sky, that's a building), SAM segments by object boundaries without caring what each thing is called. It finds the separate "things" in a scene and outlines them. It's the more experimental, more niche member of the pack's segmentation family, filed under "others" for a reason - useful when you want boundary-based structure rather than semantic labels.

How it works

The node uses MobileSAM, a lightweight distilled version of Meta's Segment Anything, so it runs fast and on modest hardware. It processes the whole image and produces an automatic segmentation - regions split along object edges - which you can then use as a control hint. Where the semantic segmentors give you the standard ADE20K color palette for a control_sd15_seg model, SAM's strength is clean boundary detection on arbitrary objects, which is why it sits apart from the ADE20K nodes.

It's worth flagging the current landscape here, because SAM segmentation had a moment in 2026: a community member trained an entirely new SAM-based ControlNet condition for Z-Image - block a composition out of flat colored shapes and generate from it - for about $200 of rented GPU. That's a different, newer artifact than what this decade-old node targets, but it shows SAM-style segmentation is having a small renaissance as a conditioning input. For the classic pack workflow, treat this node as the boundary-segmentation option.

The inputs that matter

Simple interface. It takes an image and returns an IMAGE - the SAM segmentation - for a downstream ControlNet Apply node. The only knob:

• resolution (default 512) - working size; match your render.

There's nothing to tune on the segmentation itself; MobileSAM does the work.

Installing it

ComfyUI Manager: search ComfyUI's ControlNet Auxiliary Preprocessors, install, restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/Fannovel16/comfyui_controlnet_aux
pip install -r requirements.txt

Restart ComfyUI. The MobileSAM weights (mobile_sam.pt) download from HuggingFace on first run - it's a small model, so this is quick.

Where people get burned

The honest caution: this is a niche, "others"-category node, and it's less of a plug-and-play win than the depth/pose/canny preprocessors. SAM gives you boundaries, not semantic labels, so it doesn't drop into the standard ADE20K seg ControlNet the way UniFormer and OneFormer do - be clear about what you're conditioning on and which model you're pairing it with. If you actually want category-based layout control (sky, person, building), the semantic segmentors are the right tool and this isn't. And the pack-wide constant applies: the segmentation map is just an image until a compatible ControlNet is loaded downstream to act on it.