Nodes/ComfyUI-CustomNodePacks/Mask Tracker — Motion/Propagate/Anchor/Consistency
ComfyUI Node

Mask Tracker — Motion/Propagate/Anchor/Consistency

Four video-mask engines behind one mode switch

By Code2Collapse·Created 6 months ago·Updated a day ago· 52
Mask Tracker — Motion/Propagate/Anchor/Consistency
  • mask
  • video
  • sam_model
  • masks
  • preview
  • score
  • info_json
  • metric
modemotion
camera_compensationtrue
stabilization_methodhomography
detection_modecombined
pixel_diff_enabledtrue
pixel_diff_threshold0.050
flow_enabledtrue
flow_threshold1.0
flow_algorithmfarneback
bg_sub_enabledfalse
bg_model_frames5
bg_sub_threshold0.100
hist_enabledfalse
hist_grid_size16
hist_threshold0.15
combine_methodunion
grow_pixels4
min_region_size100
temporal_smoothtrue
source_frame0
propagate_modestatic
prop_flow_threshold2.0
fade_start1.00
fade_end0.00
bidirectionaltrue
anchor_frames0
total_frames30
easingsmooth_step
sdf_iterations64
flow_refinementfalse
metricpixel_diff
binarize_threshold0.50
points_json

This pack ships a lot of video-mask machinery, and Mask Tracker (MEC) is the node that pulls four of those engines under one roof. Pick a mode, and the corresponding engine runs: per-frame motion detection, seed-mask propagation, keyframe interpolation, or a flicker consistency check. Same (mask, video) input pair for all of them, same five-port output schema. It's the "unified video-mask tracker" and it's worth reaching for before you go hunting through the individual nodes.

The four modes

  • motion (default) - per-frame motion mask from four independent detectors: pixel_diff, optical_flow (Farneback or phase correlation), background_sub, and histogram_diff. camera_compensation (default on) subtracts global camera motion via stabilization_method (homography/affine/translation) so a panning shot doesn't trigger motion everywhere. combine_method (union/intersection) merges the active detectors, then grow_pixels and min_region_size clean up the result. The output score is motion intensity. This is the mode for "what actually moved in this clip."
  • propagate - seed mask on one source_frame, push it to all frames. Methods: static (same mask everywhere), optical_flow, sam2_video (needs a wired sam_model + optional points_json), fade, scale_linear. bidirectional propagates both forward and backward from the seed. This is the rotoscope-helper.
  • anchor - SDF-based interpolation between keyframe masks. Give anchor_frames as a CSV ("0,10,30") and one mask per anchor in the mask input; it morphs smoothly between them across total_frames with your choice of easing and sdf_iterations. Optional flow_refinement warps the SDF to follow actual motion (needs video). The output score is mean confidence.
  • consistency_check - flicker scoring between consecutive frames, via metric (mask_iou / pixel_diff / flow_warp). This is the QA mode: run it on a propagated batch to find the frames that break.

Outputs

All modes share: masks (per-frame batch), preview, score (mode-specific scalar), info_json (the mode's diagnostic payload), and metric (a label string saying what produced the result). The mask input is required by propagate/anchor/consistency but optional for motion.

Where it fits

For a video cutout, propagate + SAM2 is the quality path (SAM2's video memory is genuinely good at holding identity). For change detection, motion with camera compensation is the fast, model-free path. For a clean keyframe morph without any tracking, anchor is overkill-free interpolation. And after any of them, consistency_check tells you if it actually held together - pair that with MaskTemporal for the stabilization + warning report.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/Code2Collapse/ComfyUI-CustomNodePacks.git
pip install opencv-python>=4.7.0 scipy>=1.10.0

or ComfyUI Manager → search "CustomNodePacks", restart, confirm [MEC] Loaded .... The only heavy dependency is sam2_video propagate mode, which needs a SAM 2.1 checkpoint in ComfyUI/models/sam2/. Everything else is OpenCV/torch and works on CPU. When a mode silently does less than you expected, read info_json - the node reports what actually ran.

CategoryC2C/Video

Inputs (36)

NameTypeDefaultDescription
modeCOMBOmotionmotion: per-frame motion mask (pixel/flow/bg/hist). propagate: seed mask on one frame, push to all frames. anchor: SDF interpolation between anchor masks. consistency_check: score flicker between consecutive frames.
camera_compensationBOOLEANtrue[motion] subtract global camera motion
stabilization_methodCOMBOhomography[motion] camera-motion model
detection_modeCOMBOcombined[motion] active method(s)
pixel_diff_enabledBOOLEANtrue[motion] enable pixel-diff method
pixel_diff_thresholdFLOAT0.0500.001–1[motion] pixel-diff threshold
flow_enabledBOOLEANtrue[motion] enable optical flow
flow_thresholdFLOAT1.00.1–50[motion] flow magnitude threshold
flow_algorithmCOMBOfarneback[motion] flow algorithm
bg_sub_enabledBOOLEANfalse[motion] background subtraction
bg_model_framesINT51–30[motion] frames for bg model
bg_sub_thresholdFLOAT0.1000.001–1[motion] bg-diff threshold
hist_enabledBOOLEANfalse[motion] histogram diff
hist_grid_sizeINT164–64[motion] histogram grid NxN
hist_thresholdFLOAT0.150.01–1[motion] histogram L2 threshold
combine_methodCOMBOunion[motion] method combination
grow_pixelsFLOAT40–64[motion] dilate result
min_region_sizeINT1000–10000[motion] noise filter
temporal_smoothBOOLEANtrue[motion] gaussian time smoothing
source_frameINT00–99999[propagate] frame where mask is drawn
propagate_modeCOMBOstatic[propagate] propagation method
prop_flow_thresholdFLOAT2.00–50[propagate] optical-flow threshold
fade_startFLOAT1.000–1[propagate] opacity at source frame
fade_endFLOAT0.000–1[propagate] opacity at last frame
bidirectionalBOOLEANtrue[propagate] forward+backward from source
anchor_framesSTRING0[anchor] CSV frame indices for each anchor mask
total_framesINT301–99999[anchor] total output frames
easingCOMBOsmooth_step[anchor] easing curve
sdf_iterationsINT644–512[anchor] SDF diffusion iterations
flow_refinementBOOLEANfalse[anchor] optical-flow refine (needs video)
metricCOMBOpixel_diff[consistency_check] metric
binarize_thresholdFLOAT0.500.01–0.99[consistency_check] mask binarize threshold
maskoptMASKRequired by propagate (seed), anchor (anchor stack), consistency_check (mask_iou). Optional for motion.
videooptIMAGEVideo frame batch (B,H,W,C). Required by motion, propagate, anchor flow_refinement, and pixel/flow consistency.
sam_modeloptSAM_MODEL[propagate sam2_video mode] SAM2 model
points_jsonoptSTRING[propagate sam2_video mode] point prompts

Outputs (5)

NameTypeDescription
masksMASKPer-frame mask batch (B,H,W).
previewIMAGEPreview overlay (propagate) or video passthrough.
scoreFLOATMode-specific scalar: motion intensity / mean confidence / flicker score.
info_jsonSTRINGMode-specific JSON diagnostic payload.
metricSTRINGMode/metric label string.