ComfyUI Node

Mask + Matting

The one-node production pipeline from segmentation to alpha to despill

By Code2Collapse·Created 8 months ago·Updated about 3 hours ago· 59
Mask + Matting
  • image
  • pos_bbox
  • neg_bbox
  • normal_bbox
  • external_mask
  • external_trimap
  • holdout_mask
  • core_mask
  • sam_model
  • background_plate
  • mask
  • alpha
  • preview
  • trimap
  • bbox
  • bbox_json
  • score
  • info
  • despilled
  • lightwrap_rgba
  • edge_mask
  • inside_mask
  • outside_mask
  • luma_key_mask
  • problem_regions
  • severity
  • suggested_method
  • object_alphas
  • foreground
◄segmenterauto_best►
◄mattervitmatte►
◄model(auto)►
◄matter_model(auto)►
◄precisionfp16►
◄attentionauto►
◄offloadnone►
◄subject_presetcustom►
◄trimap_dilate8►
◄trimap_erode8►
◄edge_radius4►
◄edge_modesoft►
◄edge_threshold0.50►
◄edge_feather0.0►
◄individual_objectsfalse►
◄tracking_directionforward►
◄frame_annotation0►
◄object_id0►
◄max_frames_to_track0►
◄memory_size8►
◄start_frame0►
◄end_frame-1►
◄auto_downloadfalse►
◄seed0►
◄tta_flipfalse►
◄multiscalefalse►
◄post_refinenone►
◄refine_radius8►
◄refine_iterations5►
◄despilloff►
◄despill_strength1.00►
◄preserve_skintrue►
◄lightwrap_strength0.00►
◄lightwrap_radius8►
◄edge_band_radius4►
◄premultiplytrue►
◄enable_luma_keyfalse►
◄luma_modeauto►
◄luma_low0.00►
◄luma_high1.00►
◄luma_gamma1.00►
◄luma_falloff1.0►
◄luma_invertfalse►
◄luma_mixhint_only►
◄enable_advanced_trimapfalse►
◄trimap_inner_scale1.0►
◄trimap_outer_scale1.5►
◄trimap_smooth0.0►
◄trimap_threshold0.50►
◄auto_qualitytrue►
◄auto_disambiguatetrue►
◄quality_modebalanced►
◄enable_diagnosetrue►
◄diag_ring_width5►
◄diag_blur_threshold50►
◄diag_brightness_threshold0.15►
◄robust_propagationfalse►
◄robust_confidence_threshold0.65►
◄robust_reanchor_methodblend►
◄robust_blend_alpha0.70►
◄positive_coords►
◄negative_coords►
◄text_prompt►
◄pipelineonyx►
◄scene_prompts►
◄band_scale1.00►
◄matte_tile1024►
◄temporal_stabilisetrue►
◄matte_overlap64►
◄matte_tile_batch4►
◄backendsegment + matte►
◄points_json[]►
◄bbox_json►
◄sam_iterations2►
◄sam_refine_methodauto►
◄sam_edge_radius12►
◄sam_detail_preservation0.85►
◄sam_edge_contrast1.0►
◄sam_multimask_outputtrue►
◄sam_mask_index0►
◄sam_score_threshold0.00►
◄sam_trimap_dilate0►
◄sam_trimap_erode0►
◄sam_batch_modefalse►
◄sec_modelsam2.1_hiera_base_plus►
◄sec_text_prompt►
◄sec_matting_backendauto►
◄sec_edge_radius15►
◄sec_n_warmup5►
◄sec_edge_refine_methodnone►
◄bg_modelbirefnet_general►
◄bg_threshold0.50►
◄bg_invertfalse►
◄bg_mask_blur0►
◄fill_holes_enabledtrue►
◄min_region_size64►
◄keep_model_loadedtrue►
◄split_maskstrue►

This is the flagship. Mask Ops (MEC) is a full production matting pipeline in one node - segmenter, alpha matter, auto-quality pre-processing, despill, light wrap, video propagation, and diagnostics - with 17 outputs and enough knobs to scare anyone. The good news: the defaults are so aggressively automatic that you can ignore most of them and still get a better cutout than a hand-rolled chain.

The pipeline: a coarse mask from a segmenter, then an optional alpha matte, then polish. In this pack's terms it's MaskOpsMEC = the "Mask + Matting" node - the one-stop shop where the separate SAM/ViTMatte/RVM nodes get fused.

The parts that matter

  • segmenter (default auto_best) - the coarse-mask backend: SAM 2.1, SAM 3, SAM 3.1, BiRefNet, RMBG-2.0, InSPyReNet, DIS, GroundingDINO, and more. Entries tagged [missing-deps] need an optional pip install to activate. auto_best picks what's available and best - leave it unless you know why not.
  • matter (default vitmatte) - refines the alpha: none (use the segmenter mask as-is), vitmatte, rvm, bgmattingv2, matanyone, birefnet. This is where hair and fur actually get good.
  • model / matter_model - pick specific weight files, or (auto) and let each backend choose.
  • auto_quality (default ON) - the killer feature. It detects motion blur, low light, low contrast, speckle noise, and similar fg/bg color, then applies just-enough preprocessing (CLAHE, unsharp, NL-means, chroma stretch) before segmentation and a light guided-filter polish after. No knobs. On hard images this is the difference between a decent mask and a good one.
  • auto_disambiguate (default ON) - feed positive_coords (e.g. on the face) and negative_coords (e.g. on the neck) and it scores SAM's three candidates by pos-coverage minus neg-coverage, so you get the face, not the whole person.

Then the pro stuff: despill with preserve_skin (decontaminate green/blue/whatever backing), lightwrap_strength for blending over a new background, holdout_mask (force pixels to alpha 0 - boom mics, rigs), core_mask (force pixels to alpha 1), post_refine (guided or DenseCRF), tta_flip/multiscale for quality, and robust_propagation for video - a confidence-aware re-anchor loop that re-prompts SAM when a frame's mask drifts (flow warp / DINOv2 / blend strategies).

Outputs worth wiring

The full list is long; the ones you'll actually use: alpha (the production matte), mask (coarse), preview (premultiplied), bbox/bbox_json, score (0–1 quality), info (what actually ran), and the compositing aids - despilled, lightwrap_rgba, edge_mask, inside_mask/outside_mask. problem_regions/severity/suggested_method come from the built-in failure explainer.

Install and model files

Pack install, then models:

cd ComfyUI/custom_nodes
git clone https://github.com/Code2Collapse/ComfyUI-CustomNodePacks.git
pip install opencv-python>=4.7.0 scipy>=1.10.0

SAM weights go in ComfyUI/models/sams/ (SAM 3) and ComfyUI/models/sam2/ (SAM 2.1); BiRefNet/RMBG/InSPyReNet weights are auto-listed from their standard folders. auto_download (default off) can lazy-fetch missing weights from HF/torch.hub. The [missing-deps] tags tell you exactly which pip packages unlock which backends - install transformers for ViTMatte, and the extras for the video matters.

Three gotchas: (1) RMBG-2.0 is non-commercial under BRIA's license - check before shipping a product. (2) SAM 3 is not Apache like SAM/SAM2, worth knowing for commercial use. (3) With 50+ inputs, keep auto_quality on and resist touching everything - the node's own docs describe it as "no knobs to tune," and it's right.

Category🐺 C2C/🧰 Core/Pipeline

Inputs (108)

NameTypeDefaultDescription
imageIMAGESource image or video frames (B,H,W,C).
segmenterCOMBOauto_bestCoarse-mask backend. Entries tagged [missing-deps] need an optional pip install to activate.
matterCOMBOvitmatteOptional alpha refinement. 'none' returns the segmenter mask as alpha.
modelCOMBO(auto)Specific weight file to use. Tag prefix selects the backend folder; '(auto)' lets each backend pick.
matter_modelCOMBO(auto)Weight file for the matter backend.
precisionCOMBOfp163 options: fp16, bf16, fp32
attentionCOMBOauto6 options: auto, sdpa, flash, sage, xformers, eager
offloadCOMBOnone3 options: none, cpu, sequential
subject_presetCOMBOcustomOverride trimap_dilate/erode/edge with subject-tuned values.
trimap_dilateINT80–128—
trimap_erodeINT80–128—
edge_radiusINT40–64—
edge_modeCOMBOsoftFinal alpha edge. 'soft' = as the matter produced it. 'feather' = blur the existing edge. 'hard' = binary cut at edge_threshold, no partial alpha (what a holdout or garbage matte wants). 'hard+feather' = cut hard then feather — the edge width becomes yours rather than the model's, which is how you match edges across shots keyed by different backends.
edge_thresholdFLOAT0.500–1Cut point for the hard modes. Lower keeps more of the soft fringe as solid; higher eats into it.
edge_featherFLOAT0.00–64Feather radius in pixels, read as ~2 sigma of a true Gaussian. 0 with a feather mode selected does nothing.
individual_objectsBOOLEANfalseIf supported by the backend, return one mask per detected object.
tracking_directionCOMBOforward3 options: forward, backward, bidirectional
frame_annotationINT00–100000Frame index (in clip) where prompts are anchored.
object_idINT00–1024—
max_frames_to_trackINT00–1000000 = no cap.
memory_sizeINT81–256—
start_frameINT00–100000—
end_frameINT-1-1–100000-1 = last frame.
auto_downloadBOOLEANfalseAllow Download: fetch a missing weight from Hugging Face / torch.hub into its models/ subfolder. Off: installed copies are used wherever they are (this pack, other packs, the HF cache); without one the model-free method runs and the info output says so.
seedINT00–18446744073709550000—
tta_flipBOOLEANfalseTest-time augmentation: run segmenter on the H-flipped image and average. Slower but cleaner.
multiscaleBOOLEANfalseRun the segmenter at 0.75x / 1.0x / 1.25x and fuse. Helps small / thin subjects.
post_refineCOMBOnoneFinal alpha refinement. 'guided' = guided filter (fast, torch-only). 'crf' = DenseCRF (requires pydensecrf, sharpest edges).
refine_radiusINT81–64Spatial radius for guided / CRF refinement.
refine_iterationsINT51–30CRF inference iterations.
despillCOMBOoffColour decontamination on the named backing. 'auto' estimates the colour from image corners.
despill_strengthFLOAT1.000–2How aggressively to subtract the spill (0 = off).
preserve_skinBOOLEANtrueKeep warm pixels (R>G>B) untouched during despill.
lightwrap_strengthFLOAT0.000–2Light-wrap intensity. 0 = off; ~0.3-0.6 = natural blend over the new BG.
lightwrap_radiusINT81–64Light-wrap halo radius in pixels.
edge_band_radiusINT41–64Width of the soft edge band when splitting edge/inside/outside masks.
premultiplyBOOLEANtruePremultiply preview by alpha. Disable for straight-alpha outputs.
enable_luma_keyBOOLEANfalseRun a luminance keyer on the source image BEFORE segmentation and use it as a hint / external_mask.
luma_modeCOMBOauto5 options: auto, highlights, midtones, shadows, custom
luma_lowFLOAT0.000–1—
luma_highFLOAT1.000–1—
luma_gammaFLOAT1.000.01–10—
luma_falloffFLOAT1.00–10—
luma_invertBOOLEANfalse—
luma_mixCOMBOhint_onlyHow to combine the luma-key mask with the segmenter result. 'hint_only' = use as external_mask hint; 'intersect/union/replace' = combine with the final alpha.
enable_advanced_trimapBOOLEANfalseUse the edge-aware trimap generator (asymmetric inner/outer scaling, image-edge snapping, smoothing) instead of the simple dilate/erode trimap.
trimap_inner_scaleFLOAT1.00.1–3—
trimap_outer_scaleFLOAT1.50.5–5—
trimap_smoothFLOAT0.00–20—
trimap_thresholdFLOAT0.500–1—
auto_qualityBOOLEANtrueAUTOMATIC robustness for hard images. Detects motion blur, low light, low contrast, speckle noise and low bg/fg colour separation, then applies just-enough pre-processing (CLAHE, unsharp, NL-means, chroma stretch) BEFORE the segmenter, and a light guided-filter+edge-snap polish on the alpha. No knobs.
auto_disambiguateBOOLEANtrueWhen BOTH positive and negative points are supplied, score SAM's 3 candidate masks by (pos-coverage − neg-coverage − size-penalty) instead of raw score. This is what makes `pos=face, neg=neck` return just the face, not the whole person.
quality_modeCOMBObalancedStrength of auto_quality pre/post processing. fast = mild, balanced = default, max_fidelity = NL-means denoise + larger guided filter.
enable_diagnoseBOOLEANtrueRun automatic mask-failure diagnostics (severity score + suggested method).
diag_ring_widthINT51–50—
diag_blur_thresholdFLOAT500–1000—
diag_brightness_thresholdFLOAT0.150–1—
robust_propagationBOOLEANfalseConfidence-aware re-anchor loop for video. After SAM2 propagation, every frame's mask is scored vs the last good mask (IoU × size-ratio). When confidence drops below the threshold, the chosen re-anchor strategy fires (flow warp / DINOv2 region search / convex blend). Robust against motion blur + lighting drift. Single-image inputs skip this stage.
robust_confidence_thresholdFLOAT0.650–1Below this confidence the re-anchor fires.
robust_reanchor_methodCOMBOblendflow=Farneback optical-flow warp of last good mask. dino=DINOv2 patch-feature region search + SAM2 re-prompt. blend=convex mix of current+warped. none=accept drifted output (debug).
robust_blend_alphaFLOAT0.700–1Blend weight for current SAM2 mask when method=blend.
positive_coordsoptSTRINGJSON list of positive points [[x,y],...] from PointsBBoxMaskEditor (positive_coords output).
negative_coordsoptSTRINGJSON list of negative points [[x,y],...] from PointsBBoxMaskEditor (negative_coords output).
pos_bboxoptBBOXSingle positive bbox [x0,y0,x1,y1].
neg_bboxoptBBOXOptional negative bbox (excluded region).
normal_bboxoptBBOXGeneric bbox if you don't care about polarity.
text_promptoptSTRINGOpen-vocabulary text prompt (SAM3 / GroundingDINO / VideoMaMa). Wire from any STRING source.
external_maskoptMASKOptional mask used as a hint or overridden when input_mode='auto' falls through.
external_trimapoptMASKOptional pre-computed trimap that bypasses internal trimap generation.
holdout_maskoptMASKGarbage / holdout matte. Pixels where this is >0 are FORCED to alpha=0 (used to chop out boom mics, rigs, etc).
core_maskoptMASKCore / inside matte. Pixels where this is >0 are FORCED to alpha=1 (used to lock down opaque interiors).
pipelineoptCOMBOonyxonyx = SAM3.1 video session + tiled ViTMatte. cascade (legacy) = previous multi-backend auto_best path.
scene_promptsoptSTRINGOptional ONYX scene JSON (version 1). When empty, legacy prompt sockets map to object 1.
band_scaleoptFLOAT1.000.1–5Resolution-aware trimap band scale (ONYX path).
matte_tileoptINT1024256–4096ViTMatte tile size in pixels for both the ONYX and cascade pipelines.
temporal_stabiliseoptBOOLEANtrue3-tap temporal median on alpha inside the unknown trimap band (ONYX path).
matte_overlapoptINT6416–256Overlap between ViTMatte tiles in pixels. Larger overlaps reduce seams but run more tiles.
matte_tile_batchoptINT41–16How many tiles to run per GPU forward pass. Lower this if you hit out-of-memory errors. On the CPU tiles always run one at a time (faster there, and a third of the memory).
backendoptCOMBOsegment + matteWhich pipeline makes the matte. 'segment + matte' = this node's segmenter and matter (pipeline, segmenter, matter). 'SAM + ViTMatte' = a SAM model from a SAM Model Loader, then ViTMatte or a guided filter on the edges. 'SeC + MatAnyone2' = SeC / SAM segmentation, then MatAnyone2 (video) or ViTMatte. 'remove background' = RMBG-2.0 or BiRefNet, no prompt. Each control's tooltip names the backend that reads it.
sam_modeloptSAM_MODELSAM + ViTMatte: the model from a SAM Model Loader.
points_jsonoptSTRING[]SAM + ViTMatte and SeC + MatAnyone2: point prompts: [{"x":100,"y":200,"label":1}, ...] (label 1 = keep, 0 = remove). The click editor writes this.
bbox_jsonoptSTRINGSAM + ViTMatte and SeC + MatAnyone2: box prompt [x1,y1,x2,y2]. Empty = points only. A wired pos_bbox wins.
sam_iterationsoptINT21–5SAM + ViTMatte: SAM passes; each pass uses the previous mask to place better prompts. 2-3 is ideal.
sam_refine_methodoptCOMBOautoSAM + ViTMatte: edge refinement. auto = ViTMatte, else multi-scale guided filter, else guided filter. vitmatte = neural matting. guided_filter = fast. multi_scale_guided = three scales (best without a model). color_aware = LAB colour edges. laplacian_blend = frequency blend.
sam_edge_radiusoptINT121–200SAM + ViTMatte: pixels around the edge that are refined (larger = softer transitions).
sam_detail_preservationoptFLOAT0.850–1SAM + ViTMatte: fine detail (hair, fur, lace) to keep. 0 = smooth, 1 = all.
sam_edge_contrastoptFLOAT1.00–3SAM + ViTMatte: edge contrast for hard lighting. Above 1 sharpens the boundary.
sam_multimask_outputoptBOOLEANtrueSAM + ViTMatte: SAM returns three candidate masks (off = its single best).
sam_mask_indexoptINT00–2SAM + ViTMatte: which of SAM's three candidates to keep (with multimask output on).
sam_score_thresholdoptFLOAT0.000–1SAM + ViTMatte: skip SAM candidates below this confidence (0 = use mask_index).
sam_trimap_dilateoptINT00–200SAM + ViTMatte: outer trimap radius (0 = edge radius x 1.5).
sam_trimap_erodeoptINT00–200SAM + ViTMatte: inner trimap radius (0 = edge radius x 1.0).
sam_batch_modeoptBOOLEANfalseSAM + ViTMatte: process every frame of the batch. Off = the first frame only (every output then has one frame).
sec_modeloptCOMBOsam2.1_hiera_base_plusSeC + MatAnyone2: segmentation model. SeC = video with text prompts; SAM 2 / 3 = point or box prompts.
sec_text_promptoptSTRINGSeC + MatAnyone2: what to track, e.g. 'cat' or 'person in red'. A wired text_prompt wins. Empty = point / box prompts.
sec_matting_backendoptCOMBOautoSeC + MatAnyone2: alpha matting. auto = MatAnyone2 for video (more than one frame), ViTMatte small for an image. MatAnyone2 keeps the matte steady over time; ViTMatte has the best edge per frame.
sec_edge_radiusoptINT151–200SeC + MatAnyone2: edge refinement radius in pixels.
sec_n_warmupoptINT51–30SeC + MatAnyone2: MatAnyone2 warm-up passes on the first frame (more = steadier start).
sec_edge_refine_methodoptCOMBOnoneSeC + MatAnyone2: optional edge refinement after matting. none = the matting output as it is.
bg_modeloptCOMBObirefnet_generalremove background: rmbg_2.0 = general purpose, fast. birefnet_general = detailed edges. birefnet_portrait = people.
bg_thresholdoptFLOAT0.500–1remove background: above 0 the mask is cut hard at this value (0 = keep the soft mask).
bg_invertoptBOOLEANfalseremove background: invert the mask (keep the background instead).
bg_mask_bluroptINT00–50remove background: Gaussian blur on the mask edge, in pixels (0 = none).
fill_holes_enabledoptBOOLEANtrueSAM + ViTMatte and SeC + MatAnyone2: fill holes inside the mask.
min_region_sizeoptINT640–10000SAM + ViTMatte and SeC + MatAnyone2: remove separate regions smaller than this many pixels (0 = keep all).
keep_model_loadedoptBOOLEANtrueSeC + MatAnyone2 and remove background: keep the models in memory between runs (off = free them after each run).
split_masksoptBOOLEANtrueMake the edge / inside / outside masks (and, for the three other backends, a trimap from the final alpha). Off leaves those outputs empty, which saves three to four full-size masks per frame on long clips.
background_plateoptIMAGEmatter = bgmattingv2 only: the same shot WITHOUT the subject (a clean plate). One frame serves every frame; a plate batch as long as the clip is used frame by frame; a different size is resized to the image. Not connected: a plate is synthesised from the coarse mask, which is much less accurate.

Outputs (19)

NameTypeDescription
maskMASKCoarse mask from the segmenter (B,H,W).
alphaMASKRefined alpha after matter + refinement (production output).
previewIMAGEimage * alpha premultiplied preview.
trimapMASKTrimap (0/0.5/1) used by the matter.
bboxBBOXTight bbox around the alpha as [x0,y0,x1,y1].
bbox_jsonSTRINGSame bbox as JSON {'x','y','w','h'}.
scoreFLOATOverall production-quality score in [0,1] (boundary + coherence + size + smoothness).
infoSTRINGJSON: backends, modes, per-frame quality breakdown, refine stages run, settings used.
despilledIMAGEImage with backing-colour spill suppressed (when `despill_strength`>0).
lightwrap_rgbaIMAGERGBA light-wrap layer to ADD over the new background.
edge_maskMASKSoft edge band where matting actually matters.
inside_maskMASKSolid-fg interior mask (safe to colour-grade).
outside_maskMASKSolid-bg exterior mask (safe to defocus / replace).
luma_key_maskMASKLuminance keyer output (empty if `enable_luma_key` is off).
problem_regionsMASKDiagnostic problem-region heatmap from the failure explainer.
severityFLOATSeverity score [0,1] from the failure explainer.
suggested_methodSTRINGSuggested next masking method (string) from the failure explainer.
object_alphasMASKPer-object alpha mattes stacked object-major as (O*B,H,W).
foregroundIMAGEThe image times the final alpha (premultiplied), despilled when despill is on.