Mask + Matting
The one-node production pipeline from segmentation to alpha to despill
- image
- pos_bbox
- neg_bbox
- normal_bbox
- external_mask
- external_trimap
- holdout_mask
- core_mask
- sam_model
- background_plate
- mask
- alpha
- preview
- trimap
- bbox
- bbox_json
- score
- info
- despilled
- lightwrap_rgba
- edge_mask
- inside_mask
- outside_mask
- luma_key_mask
- problem_regions
- severity
- suggested_method
- object_alphas
- foreground
This is the flagship. Mask Ops (MEC) is a full production matting pipeline in one node - segmenter, alpha matter, auto-quality pre-processing, despill, light wrap, video propagation, and diagnostics - with 17 outputs and enough knobs to scare anyone. The good news: the defaults are so aggressively automatic that you can ignore most of them and still get a better cutout than a hand-rolled chain.
The pipeline: a coarse mask from a segmenter, then an optional alpha matte, then polish. In this pack's terms it's MaskOpsMEC = the "Mask + Matting" node - the one-stop shop where the separate SAM/ViTMatte/RVM nodes get fused.
The parts that matter
segmenter(defaultauto_best) - the coarse-mask backend: SAM 2.1, SAM 3, SAM 3.1, BiRefNet, RMBG-2.0, InSPyReNet, DIS, GroundingDINO, and more. Entries tagged[missing-deps]need an optional pip install to activate.auto_bestpicks what's available and best - leave it unless you know why not.matter(defaultvitmatte) - refines the alpha:none(use the segmenter mask as-is),vitmatte,rvm,bgmattingv2,matanyone,birefnet. This is where hair and fur actually get good.model/matter_model- pick specific weight files, or(auto)and let each backend choose.auto_quality(default ON) - the killer feature. It detects motion blur, low light, low contrast, speckle noise, and similar fg/bg color, then applies just-enough preprocessing (CLAHE, unsharp, NL-means, chroma stretch) before segmentation and a light guided-filter polish after. No knobs. On hard images this is the difference between a decent mask and a good one.auto_disambiguate(default ON) - feedpositive_coords(e.g. on the face) andnegative_coords(e.g. on the neck) and it scores SAM's three candidates by pos-coverage minus neg-coverage, so you get the face, not the whole person.
Then the pro stuff: despill with preserve_skin (decontaminate green/blue/whatever backing), lightwrap_strength for blending over a new background, holdout_mask (force pixels to alpha 0 - boom mics, rigs), core_mask (force pixels to alpha 1), post_refine (guided or DenseCRF), tta_flip/multiscale for quality, and robust_propagation for video - a confidence-aware re-anchor loop that re-prompts SAM when a frame's mask drifts (flow warp / DINOv2 / blend strategies).
Outputs worth wiring
The full list is long; the ones you'll actually use: alpha (the production matte), mask (coarse), preview (premultiplied), bbox/bbox_json, score (0–1 quality), info (what actually ran), and the compositing aids - despilled, lightwrap_rgba, edge_mask, inside_mask/outside_mask. problem_regions/severity/suggested_method come from the built-in failure explainer.
Install and model files
Pack install, then models:
cd ComfyUI/custom_nodes
git clone https://github.com/Code2Collapse/ComfyUI-CustomNodePacks.git
pip install opencv-python>=4.7.0 scipy>=1.10.0
SAM weights go in ComfyUI/models/sams/ (SAM 3) and ComfyUI/models/sam2/ (SAM 2.1); BiRefNet/RMBG/InSPyReNet weights are auto-listed from their standard folders. auto_download (default off) can lazy-fetch missing weights from HF/torch.hub. The [missing-deps] tags tell you exactly which pip packages unlock which backends - install transformers for ViTMatte, and the extras for the video matters.
Three gotchas: (1) RMBG-2.0 is non-commercial under BRIA's license - check before shipping a product. (2) SAM 3 is not Apache like SAM/SAM2, worth knowing for commercial use. (3) With 50+ inputs, keep auto_quality on and resist touching everything - the node's own docs describe it as "no knobs to tune," and it's right.
Inputs (108)
| Name | Type | Default | Description |
|---|---|---|---|
| image | IMAGE | Source image or video frames (B,H,W,C). | |
| segmenter | COMBO | auto_best | Coarse-mask backend. Entries tagged [missing-deps] need an optional pip install to activate. |
| matter | COMBO | vitmatte | Optional alpha refinement. 'none' returns the segmenter mask as alpha. |
| model | COMBO | (auto) | Specific weight file to use. Tag prefix selects the backend folder; '(auto)' lets each backend pick. |
| matter_model | COMBO | (auto) | Weight file for the matter backend. |
| precision | COMBO | fp16 | 3 options: fp16, bf16, fp32 |
| attention | COMBO | auto | 6 options: auto, sdpa, flash, sage, xformers, eager |
| offload | COMBO | none | 3 options: none, cpu, sequential |
| subject_preset | COMBO | custom | Override trimap_dilate/erode/edge with subject-tuned values. |
| trimap_dilate | INT | 80–128 | — |
| trimap_erode | INT | 80–128 | — |
| edge_radius | INT | 40–64 | — |
| edge_mode | COMBO | soft | Final alpha edge. 'soft' = as the matter produced it. 'feather' = blur the existing edge. 'hard' = binary cut at edge_threshold, no partial alpha (what a holdout or garbage matte wants). 'hard+feather' = cut hard then feather — the edge width becomes yours rather than the model's, which is how you match edges across shots keyed by different backends. |
| edge_threshold | FLOAT | 0.500–1 | Cut point for the hard modes. Lower keeps more of the soft fringe as solid; higher eats into it. |
| edge_feather | FLOAT | 0.00–64 | Feather radius in pixels, read as ~2 sigma of a true Gaussian. 0 with a feather mode selected does nothing. |
| individual_objects | BOOLEAN | false | If supported by the backend, return one mask per detected object. |
| tracking_direction | COMBO | forward | 3 options: forward, backward, bidirectional |
| frame_annotation | INT | 00–100000 | Frame index (in clip) where prompts are anchored. |
| object_id | INT | 00–1024 | — |
| max_frames_to_track | INT | 00–100000 | 0 = no cap. |
| memory_size | INT | 81–256 | — |
| start_frame | INT | 00–100000 | — |
| end_frame | INT | -1-1–100000 | -1 = last frame. |
| auto_download | BOOLEAN | false | Allow Download: fetch a missing weight from Hugging Face / torch.hub into its models/ subfolder. Off: installed copies are used wherever they are (this pack, other packs, the HF cache); without one the model-free method runs and the info output says so. |
| seed | INT | 00–18446744073709550000 | — |
| tta_flip | BOOLEAN | false | Test-time augmentation: run segmenter on the H-flipped image and average. Slower but cleaner. |
| multiscale | BOOLEAN | false | Run the segmenter at 0.75x / 1.0x / 1.25x and fuse. Helps small / thin subjects. |
| post_refine | COMBO | none | Final alpha refinement. 'guided' = guided filter (fast, torch-only). 'crf' = DenseCRF (requires pydensecrf, sharpest edges). |
| refine_radius | INT | 81–64 | Spatial radius for guided / CRF refinement. |
| refine_iterations | INT | 51–30 | CRF inference iterations. |
| despill | COMBO | off | Colour decontamination on the named backing. 'auto' estimates the colour from image corners. |
| despill_strength | FLOAT | 1.000–2 | How aggressively to subtract the spill (0 = off). |
| preserve_skin | BOOLEAN | true | Keep warm pixels (R>G>B) untouched during despill. |
| lightwrap_strength | FLOAT | 0.000–2 | Light-wrap intensity. 0 = off; ~0.3-0.6 = natural blend over the new BG. |
| lightwrap_radius | INT | 81–64 | Light-wrap halo radius in pixels. |
| edge_band_radius | INT | 41–64 | Width of the soft edge band when splitting edge/inside/outside masks. |
| premultiply | BOOLEAN | true | Premultiply preview by alpha. Disable for straight-alpha outputs. |
| enable_luma_key | BOOLEAN | false | Run a luminance keyer on the source image BEFORE segmentation and use it as a hint / external_mask. |
| luma_mode | COMBO | auto | 5 options: auto, highlights, midtones, shadows, custom |
| luma_low | FLOAT | 0.000–1 | — |
| luma_high | FLOAT | 1.000–1 | — |
| luma_gamma | FLOAT | 1.000.01–10 | — |
| luma_falloff | FLOAT | 1.00–10 | — |
| luma_invert | BOOLEAN | false | — |
| luma_mix | COMBO | hint_only | How to combine the luma-key mask with the segmenter result. 'hint_only' = use as external_mask hint; 'intersect/union/replace' = combine with the final alpha. |
| enable_advanced_trimap | BOOLEAN | false | Use the edge-aware trimap generator (asymmetric inner/outer scaling, image-edge snapping, smoothing) instead of the simple dilate/erode trimap. |
| trimap_inner_scale | FLOAT | 1.00.1–3 | — |
| trimap_outer_scale | FLOAT | 1.50.5–5 | — |
| trimap_smooth | FLOAT | 0.00–20 | — |
| trimap_threshold | FLOAT | 0.500–1 | — |
| auto_quality | BOOLEAN | true | AUTOMATIC robustness for hard images. Detects motion blur, low light, low contrast, speckle noise and low bg/fg colour separation, then applies just-enough pre-processing (CLAHE, unsharp, NL-means, chroma stretch) BEFORE the segmenter, and a light guided-filter+edge-snap polish on the alpha. No knobs. |
| auto_disambiguate | BOOLEAN | true | When BOTH positive and negative points are supplied, score SAM's 3 candidate masks by (pos-coverage − neg-coverage − size-penalty) instead of raw score. This is what makes `pos=face, neg=neck` return just the face, not the whole person. |
| quality_mode | COMBO | balanced | Strength of auto_quality pre/post processing. fast = mild, balanced = default, max_fidelity = NL-means denoise + larger guided filter. |
| enable_diagnose | BOOLEAN | true | Run automatic mask-failure diagnostics (severity score + suggested method). |
| diag_ring_width | INT | 51–50 | — |
| diag_blur_threshold | FLOAT | 500–1000 | — |
| diag_brightness_threshold | FLOAT | 0.150–1 | — |
| robust_propagation | BOOLEAN | false | Confidence-aware re-anchor loop for video. After SAM2 propagation, every frame's mask is scored vs the last good mask (IoU × size-ratio). When confidence drops below the threshold, the chosen re-anchor strategy fires (flow warp / DINOv2 region search / convex blend). Robust against motion blur + lighting drift. Single-image inputs skip this stage. |
| robust_confidence_threshold | FLOAT | 0.650–1 | Below this confidence the re-anchor fires. |
| robust_reanchor_method | COMBO | blend | flow=Farneback optical-flow warp of last good mask. dino=DINOv2 patch-feature region search + SAM2 re-prompt. blend=convex mix of current+warped. none=accept drifted output (debug). |
| robust_blend_alpha | FLOAT | 0.700–1 | Blend weight for current SAM2 mask when method=blend. |
| positive_coordsopt | STRING | JSON list of positive points [[x,y],...] from PointsBBoxMaskEditor (positive_coords output). | |
| negative_coordsopt | STRING | JSON list of negative points [[x,y],...] from PointsBBoxMaskEditor (negative_coords output). | |
| pos_bboxopt | BBOX | Single positive bbox [x0,y0,x1,y1]. | |
| neg_bboxopt | BBOX | Optional negative bbox (excluded region). | |
| normal_bboxopt | BBOX | Generic bbox if you don't care about polarity. | |
| text_promptopt | STRING | Open-vocabulary text prompt (SAM3 / GroundingDINO / VideoMaMa). Wire from any STRING source. | |
| external_maskopt | MASK | Optional mask used as a hint or overridden when input_mode='auto' falls through. | |
| external_trimapopt | MASK | Optional pre-computed trimap that bypasses internal trimap generation. | |
| holdout_maskopt | MASK | Garbage / holdout matte. Pixels where this is >0 are FORCED to alpha=0 (used to chop out boom mics, rigs, etc). | |
| core_maskopt | MASK | Core / inside matte. Pixels where this is >0 are FORCED to alpha=1 (used to lock down opaque interiors). | |
| pipelineopt | COMBO | onyx | onyx = SAM3.1 video session + tiled ViTMatte. cascade (legacy) = previous multi-backend auto_best path. |
| scene_promptsopt | STRING | Optional ONYX scene JSON (version 1). When empty, legacy prompt sockets map to object 1. | |
| band_scaleopt | FLOAT | 1.000.1–5 | Resolution-aware trimap band scale (ONYX path). |
| matte_tileopt | INT | 1024256–4096 | ViTMatte tile size in pixels for both the ONYX and cascade pipelines. |
| temporal_stabiliseopt | BOOLEAN | true | 3-tap temporal median on alpha inside the unknown trimap band (ONYX path). |
| matte_overlapopt | INT | 6416–256 | Overlap between ViTMatte tiles in pixels. Larger overlaps reduce seams but run more tiles. |
| matte_tile_batchopt | INT | 41–16 | How many tiles to run per GPU forward pass. Lower this if you hit out-of-memory errors. On the CPU tiles always run one at a time (faster there, and a third of the memory). |
| backendopt | COMBO | segment + matte | Which pipeline makes the matte. 'segment + matte' = this node's segmenter and matter (pipeline, segmenter, matter). 'SAM + ViTMatte' = a SAM model from a SAM Model Loader, then ViTMatte or a guided filter on the edges. 'SeC + MatAnyone2' = SeC / SAM segmentation, then MatAnyone2 (video) or ViTMatte. 'remove background' = RMBG-2.0 or BiRefNet, no prompt. Each control's tooltip names the backend that reads it. |
| sam_modelopt | SAM_MODEL | SAM + ViTMatte: the model from a SAM Model Loader. | |
| points_jsonopt | STRING | [] | SAM + ViTMatte and SeC + MatAnyone2: point prompts: [{"x":100,"y":200,"label":1}, ...] (label 1 = keep, 0 = remove). The click editor writes this. |
| bbox_jsonopt | STRING | SAM + ViTMatte and SeC + MatAnyone2: box prompt [x1,y1,x2,y2]. Empty = points only. A wired pos_bbox wins. | |
| sam_iterationsopt | INT | 21–5 | SAM + ViTMatte: SAM passes; each pass uses the previous mask to place better prompts. 2-3 is ideal. |
| sam_refine_methodopt | COMBO | auto | SAM + ViTMatte: edge refinement. auto = ViTMatte, else multi-scale guided filter, else guided filter. vitmatte = neural matting. guided_filter = fast. multi_scale_guided = three scales (best without a model). color_aware = LAB colour edges. laplacian_blend = frequency blend. |
| sam_edge_radiusopt | INT | 121–200 | SAM + ViTMatte: pixels around the edge that are refined (larger = softer transitions). |
| sam_detail_preservationopt | FLOAT | 0.850–1 | SAM + ViTMatte: fine detail (hair, fur, lace) to keep. 0 = smooth, 1 = all. |
| sam_edge_contrastopt | FLOAT | 1.00–3 | SAM + ViTMatte: edge contrast for hard lighting. Above 1 sharpens the boundary. |
| sam_multimask_outputopt | BOOLEAN | true | SAM + ViTMatte: SAM returns three candidate masks (off = its single best). |
| sam_mask_indexopt | INT | 00–2 | SAM + ViTMatte: which of SAM's three candidates to keep (with multimask output on). |
| sam_score_thresholdopt | FLOAT | 0.000–1 | SAM + ViTMatte: skip SAM candidates below this confidence (0 = use mask_index). |
| sam_trimap_dilateopt | INT | 00–200 | SAM + ViTMatte: outer trimap radius (0 = edge radius x 1.5). |
| sam_trimap_erodeopt | INT | 00–200 | SAM + ViTMatte: inner trimap radius (0 = edge radius x 1.0). |
| sam_batch_modeopt | BOOLEAN | false | SAM + ViTMatte: process every frame of the batch. Off = the first frame only (every output then has one frame). |
| sec_modelopt | COMBO | sam2.1_hiera_base_plus | SeC + MatAnyone2: segmentation model. SeC = video with text prompts; SAM 2 / 3 = point or box prompts. |
| sec_text_promptopt | STRING | SeC + MatAnyone2: what to track, e.g. 'cat' or 'person in red'. A wired text_prompt wins. Empty = point / box prompts. | |
| sec_matting_backendopt | COMBO | auto | SeC + MatAnyone2: alpha matting. auto = MatAnyone2 for video (more than one frame), ViTMatte small for an image. MatAnyone2 keeps the matte steady over time; ViTMatte has the best edge per frame. |
| sec_edge_radiusopt | INT | 151–200 | SeC + MatAnyone2: edge refinement radius in pixels. |
| sec_n_warmupopt | INT | 51–30 | SeC + MatAnyone2: MatAnyone2 warm-up passes on the first frame (more = steadier start). |
| sec_edge_refine_methodopt | COMBO | none | SeC + MatAnyone2: optional edge refinement after matting. none = the matting output as it is. |
| bg_modelopt | COMBO | birefnet_general | remove background: rmbg_2.0 = general purpose, fast. birefnet_general = detailed edges. birefnet_portrait = people. |
| bg_thresholdopt | FLOAT | 0.500–1 | remove background: above 0 the mask is cut hard at this value (0 = keep the soft mask). |
| bg_invertopt | BOOLEAN | false | remove background: invert the mask (keep the background instead). |
| bg_mask_bluropt | INT | 00–50 | remove background: Gaussian blur on the mask edge, in pixels (0 = none). |
| fill_holes_enabledopt | BOOLEAN | true | SAM + ViTMatte and SeC + MatAnyone2: fill holes inside the mask. |
| min_region_sizeopt | INT | 640–10000 | SAM + ViTMatte and SeC + MatAnyone2: remove separate regions smaller than this many pixels (0 = keep all). |
| keep_model_loadedopt | BOOLEAN | true | SeC + MatAnyone2 and remove background: keep the models in memory between runs (off = free them after each run). |
| split_masksopt | BOOLEAN | true | Make the edge / inside / outside masks (and, for the three other backends, a trimap from the final alpha). Off leaves those outputs empty, which saves three to four full-size masks per frame on long clips. |
| background_plateopt | IMAGE | matter = bgmattingv2 only: the same shot WITHOUT the subject (a clean plate). One frame serves every frame; a plate batch as long as the clip is used frame by frame; a different size is resized to the image. Not connected: a plate is synthesised from the coarse mask, which is much less accurate. |
Outputs (19)
| Name | Type | Description |
|---|---|---|
| mask | MASK | Coarse mask from the segmenter (B,H,W). |
| alpha | MASK | Refined alpha after matter + refinement (production output). |
| preview | IMAGE | image * alpha premultiplied preview. |
| trimap | MASK | Trimap (0/0.5/1) used by the matter. |
| bbox | BBOX | Tight bbox around the alpha as [x0,y0,x1,y1]. |
| bbox_json | STRING | Same bbox as JSON {'x','y','w','h'}. |
| score | FLOAT | Overall production-quality score in [0,1] (boundary + coherence + size + smoothness). |
| info | STRING | JSON: backends, modes, per-frame quality breakdown, refine stages run, settings used. |
| despilled | IMAGE | Image with backing-colour spill suppressed (when `despill_strength`>0). |
| lightwrap_rgba | IMAGE | RGBA light-wrap layer to ADD over the new background. |
| edge_mask | MASK | Soft edge band where matting actually matters. |
| inside_mask | MASK | Solid-fg interior mask (safe to colour-grade). |
| outside_mask | MASK | Solid-bg exterior mask (safe to defocus / replace). |
| luma_key_mask | MASK | Luminance keyer output (empty if `enable_luma_key` is off). |
| problem_regions | MASK | Diagnostic problem-region heatmap from the failure explainer. |
| severity | FLOAT | Severity score [0,1] from the failure explainer. |
| suggested_method | STRING | Suggested next masking method (string) from the failure explainer. |
| object_alphas | MASK | Per-object alpha mattes stacked object-major as (O*B,H,W). |
| foreground | IMAGE | The image times the final alpha (premultiplied), despilled when despill is on. |