Nodes/ComfyUI Impact Pack/SAMDetector (segmented)
ComfyUI Node Runs on cloud

SAMDetector (segmented)

SAM masks from your detections, kept separate

By ltdrdata·Created 3 years ago·Updated 4 months ago· 3,242
SAMDetector (segmented)
  • sam_model
  • segs
  • image
  • combined_mask
  • batch_masks
detection_hint
dilation0
threshold0.93
bbox_expansion0
mask_hint_threshold0.70
mask_hint_use_negative

SAMDetector (segmented) uses Meta's Segment Anything Model to carve precise masks out of the regions you already detected - and unlike its "combined" sibling, it keeps those masks separate instead of merging them into one. You feed it a rough location (your SEGS, usually from a bounding-box detector) and SAM tightens that into an accurate silhouette that hugs the actual subject. The "segmented" part means you get the results split apart, which is what you want when you're going to treat each region differently.

Why bother? Because a box is a blunt instrument. Impact Pack's core move is detect-crop-refine, and the quality of the crop depends on the quality of the mask. A rectangle around a face includes background corners; when you refine and composite that back, the seams show. SAM traces the real outline instead, so the refined region blends into the original with far less visible edge. This is the same reason the community moved toward segmentation-shaped masks generally - the mask following the subject's contour is what kills the tell-tale box seam. The README's own showcase uses exactly this: detect a face with a bbox detector, then use a SAM detector to find the segment inside it, giving a mask that's precise instead of rectangular.

The inputs that matter

There are several knobs, but the ones that move the needle:

  • sam_model - a loaded SAM model (from Impact's SAMLoader). Required; this is the engine.
  • segs - the regions to refine into masks. SAM doesn't scan the whole image blindly here; it works from your detections.
  • image - the picture the masks are cut from.
  • detection_hint (an enum: center-1, horizontal-2, rect-4, mask-area, and so on) - how SAM is prompted inside each region. This tells SAM where to sample its point/box prompts. center-1 (a single center point) is a sane default; the multi-point and rect hints help on awkward shapes. If a mask comes out wrong, this is a good knob to try.
  • dilation and bbox_expansion - grow the mask or the search box outward for a little margin.
  • threshold (default 0.93) - SAM's confidence cutoff for what counts as part of the segment.

Two outputs: combined_mask, a single unified mask of everything, and batch_masks, the detected segments grouped as a batch so you can work with them separately. Heads up from the pack's own docs: batch_masks groups segments in arbitrary sets (the maintainer notes they're currently grouped roughly in threes and may not be fully separated), so it "provides some level of separation" rather than one-perfect-mask-per-subject. If you need clean per-subject masks, don't assume this hands them to you pristine - verify.

When to reach for the "segmented" version

Use combined when you just want one accurate mask of everything SAM found. Use segmented when the point is to handle regions independently downstream - different treatment per segment, or feeding a batch into something that iterates. If you only ever merge them again, the combined node is simpler.

Common issues

First: you need a SAM model. Impact Pack auto-downloads one (sam_vit_b) into ComfyUI/models/sams on first run, but if your SAMLoader dropdown is empty, that's the missing piece - grab a SAM checkpoint. Second: bad SEGS in, bad masks out. SAM refines what you give it; if the upstream detector missed the subject, SAM can't invent it. Third, per above: don't over-trust batch_masks to be perfectly separated - it's an approximation the author flags as a work in progress. And if a mask latches onto the wrong thing inside a region, change the detection_hint before touching anything else; it's the setting that most directly controls what SAM grabs.

Installing it

The node ships with ComfyUI Impact Pack. Install via ComfyUI-Manager (search ComfyUI Impact Pack, Install, restart), or manually: cd ComfyUI/custom_nodes && git clone https://github.com/ltdrdata/ComfyUI-Impact-Pack, install its requirements.txt in ComfyUI's Python environment (portable: ..\..\..\python_embeded\python.exe -m pip install -r requirements.txt), restart. On first run the pack auto-downloads a SAM model to ComfyUI/models/sams - that's the dependency this node lives on. To generate the SEGS it consumes, you'll usually also want a bounding-box detector; the YOLO UltralyticsDetectorProvider now lives in the separate ComfyUI-Impact-Subpack since v8.0. Impact Pack is ltdrdata's, the ComfyUI-Manager maintainer, so it's actively kept and trustworthy.

CategoryImpactPack/Detector

Inputs (9)

NameTypeDefaultDescription
sam_modelSAM_MODELSegment Anything Model for Silhouette Detection. Be sure to use the SAM_MODEL loaded through the SAMLoader (Impact) node as input.
segsSEGSThis is the segment information detected by the detector. For the SEGS region, the masks detected by SAM (Segment Anything) are created as a unified mask and a batch of individual masks.
imageIMAGEIt is assumed that segs contains only the information about the detected areas, and does not include the image. SAM (Segment Anything) operates by referencing this image.
detection_hintCOMBOIt is recommended to use only center-1. When refining the mask of SEGS with the SAM (Segment Anything) model, center-1 uses only the rectangular area of SEGS and a single point at the exact center as hints. Other options were added during the experimental stage and do not work well.
dilationINT0-512–512Set the value to dilate the result mask. If the value is negative, it erodes the mask.
thresholdFLOAT0.930–1
bbox_expansionINT00–1000When performing SAM (Segment Anything) detection within the SEGS area, the rectangular area of SEGS is expanded and used as a hint.
mask_hint_thresholdFLOAT0.700–1When detection_hint is mask-area, the mask of SEGS is used as a point hint for SAM (Segment Anything). In this case, only the areas of the mask with brightness values equal to or greater than mask_hint_threshold are used as hints.
mask_hint_use_negativeCOMBOWhen detecting with SAM (Segment Anything), negative hints are applied as follows: Small: When the SEGS is smaller than 10 pixels in size Outer: Sampling the image area outside the SEGS region at regular intervals

Outputs (2)

NameTypeDescription
combined_maskMASK
batch_masksMASK