Nodes/ComfyUI Impact Pack/SAMDetector (combined)
ComfyUI Node Runs on cloud

SAMDetector (combined)

Snap a rough box down to the real silhouette

By ltdrdata·Created 3 years ago·Updated 4 months ago· 3,242
SAMDetector (combined)
  • sam_model
  • segs
  • image
  • MASK
detection_hint
dilation0
threshold0.93
bbox_expansion0
mask_hint_threshold0.70
mask_hint_use_negative

SAMDetector (combined) takes the rough regions a bbox detector found and uses Segment Anything to turn each one into a mask that follows the actual outline of the thing. Boxes in, one clean silhouette-shaped mask out. It's the node that fixes the biggest weakness of box-based detection: a rectangle around a face also grabs a lot of not-face, and when you detail or composite through that rectangle, the edges show.

The canonical use is the combination from the Impact Pack's own examples: a BBOX detector finds the face and hands you SEGS, then this node runs SAM at each detected location to extract the precise segment - the face-shaped region, not the box. Intersect the two (a bbox mask AND a SAM silhouette) and you get a mask that's both correctly placed and correctly shaped. Feed that into a detailer and the fix aligns to the face's contour without bleeding into the background.

How it works

Segment Anything (SAM) is a promptable segmentation model: give it a point or a box on an image and it returns the object at that spot. This node walks the incoming SEGS, uses each detection as a prompt for SAM, and merges all the resulting segments into one unified MASK. "Combined" means exactly that - every detected region collapses into a single mask (the Segmented sibling keeps them separate). The detection_hint controls how the box is translated into SAM prompts - center point, a few points, the rectangle, and so on - which nudges SAM toward the right object when a plain center point would be ambiguous.

The inputs and outputs that matter

  • sam_model (required) - a loaded SAM model, from SAMLoader (Impact). This is the model that does the segmenting.
  • segs (required) - the detected regions to segment, from a BBOX/SEGM detector.
  • image (required) - the picture SAM looks at.
  • detection_hint (default center-1) - how each detection becomes a SAM prompt. center-1 (one center point) is fine for compact subjects like faces; the multi-point and rect options help when the center point lands on the wrong object.
  • dilation (default 0) - grows or shrinks the final mask. Negatives erode.
  • threshold (default 0.93) - SAM's confidence gate for including a segment. High by design; lower it if SAM is being too conservative and clipping the subject.
  • bbox_expansion (default 0) - pads the box SAM is prompted with, useful when the detector boxed the subject too tight.
  • mask_hint_threshold (default 0.7) and mask_hint_use_negative - advanced tuning for using the existing mask as a positive/negative hint. Defaults are fine to start.

Single output: MASK - one unified silhouette mask, ready for an inpaint node, a mask operation (like intersecting with the bbox mask), or a detailer's mask input.

How to install it

Install the pack via ComfyUI Manager (search ComfyUI Impact Pack, install, restart), or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/ltdrdata/ComfyUI-Impact-Pack comfyui-impact-pack
cd comfyui-impact-pack
python -m pip install -r requirements.txt   # ComfyUI's python; python_embeded on portable

then restart. The SAM weights (sam_vit_b_01ec64.pth) auto-download to ComfyUI/models/sams on first install; load them with SAMLoader (Impact). The SEGS feeding this node still needs a detector, which means the Impact Subpack for UltralyticsDetectorProvider. Linux: libgl1-mesa-glx and libglib2.0-0 for OpenCV.

Common issues & troubleshooting

The mask grabbed the wrong thing. A single center point can land SAM on a background object or the wrong part of the subject. Switch detection_hint to a multi-point option (rect-4, diamond-4) so SAM has more to go on, or tighten the upstream detector so the boxes are cleaner.

The silhouette is clipping the subject's edges. Lower threshold a little, or add a few pixels of dilation / bbox_expansion. SAM at 0.93 is deliberately strict.

It's out-of-memory or slow. SAM is a real model on top of your generation stack. If VRAM is tight, use the smaller ViT-B weights (the default) rather than ViT-H, and remember you can run the SAM editor on CPU via impact-pack.ini if you only need it occasionally.

Do I even need SAM here? Only when mask shape matters. For a face that's going to be feathered heavily anyway, the box mask from the detector alone is usually enough - reach for SAM when seams or background bleed are the actual problem.

CategoryImpactPack/Detector

Inputs (9)

NameTypeDefaultDescription
sam_modelSAM_MODELSegment Anything Model for Silhouette Detection. Be sure to use the SAM_MODEL loaded through the SAMLoader (Impact) node as input.
segsSEGSThis is the segment information detected by the detector. It refines the Mask through the SAM (Segment Anything) detector for all areas pointed to by SEGS, and combines all Masks to return as a single Mask.
imageIMAGEIt is assumed that segs contains only the information about the detected areas, and does not include the image. SAM (Segment Anything) operates by referencing this image.
detection_hintCOMBOIt is recommended to use only center-1. When refining the mask of SEGS with the SAM (Segment Anything) model, center-1 uses only the rectangular area of SEGS and a single point at the exact center as hints. Other options were added during the experimental stage and do not work well.
dilationINT0-512–512Set the value to dilate the result mask. If the value is negative, it erodes the mask.
thresholdFLOAT0.930–1Set the sensitivity threshold for the mask detected by SAM (Segment Anything). A higher value generates a more specific mask with a narrower range. For example, when pointing to a person's area, it might detect clothes, which is a narrower range, instead of the entire person.
bbox_expansionINT00–1000When performing SAM (Segment Anything) detection within the SEGS area, the rectangular area of SEGS is expanded and used as a hint.
mask_hint_thresholdFLOAT0.700–1When detection_hint is mask-area, the mask of SEGS is used as a point hint for SAM (Segment Anything). In this case, only the areas of the mask with brightness values equal to or greater than mask_hint_threshold are used as hints.
mask_hint_use_negativeCOMBOWhen detecting with SAM (Segment Anything), negative hints are applied as follows: Small: When the SEGS is smaller than 10 pixels in size Outer: Sampling the image area outside the SEGS region at regular intervals

Outputs (1)

NameTypeDescription
MASKMASK