ComfyUI Node

AD_sam_Crop

Track any object through a video with SAM3, then crop it frame-by-frame

By cardenluo·Created 2 years ago·Updated a day ago· 320
AD_sam_Crop
  • image
  • crop_img
  • transform
  • track_data
  • masks
detection_threshold0.50
max_objects4
detect_interval1
ckpt_namesam3.1_multiplex_fp16.safetensors
pos
crop_factor3.0
crop_width512
crop_height384
smoothing_presetbalanced

"Describe the object, and I'll track it through the whole clip, cropping it into a stable, same-size box on every frame." That's the pitch for AD_sam_Crop, and it's the first node in the pack's track-and-edit pipeline. It runs SAM3 video object tracking over your footage, then produces two things the rest of the pipeline needs: a stack of fixed-size per-frame crops (crop_img) and the tracking metadata (transform) that lets the downstream nodes paste edits back into the right place.

It sits at the front of a pipeline that looks like this: AD_sam_Crop → (feed the crops to AD_Inject_Latent to re-generate that region) → AD_sam_stitch (paste the refined crops back). If you've ever re-generated a face in a video and watched it swim, it's because the crops weren't stabilized - this node fixes that by smoothing the track before it ever reaches the sampler.

The inputs that matter

  • pos (STRING) - the object description. This is the key beginner input and the one people get wrong. It's a text prompt describing the object to track (e.g. "a man in a red jacket"), which gets CLIP-encoded into the conditioning that drives SAM3. Empty pos = instant failure.
  • image (IMAGE) - the frames, e.g. the IMAGE output of AD_In_VideoSplit.
  • ckpt_name - the SAM3 checkpoint, default sam3.1_multiplex_fp16.safetensors. It loads through CheckpointLoaderSimple, so the file goes in ComfyUI/models/checkpoints, not a SAM-specific folder. The model+prompt pair is cached, so re-runs with the same settings are fast.
  • detection_threshold (FLOAT, default 0.5) - how confident SAM3 must be to accept a detection. Lower it when tracking is missing frames.
  • max_objects (INT, default 4) - caps how many object instances are tracked.
  • detect_interval (INT, default 1) - how many frames between detections (the track is interpolated between them).
  • crop_factor (FLOAT, default 3) - how much padding around the object the crop keeps. 3 = the crop is 3× the object's bounding box. More context for the sampler, more chance of drift.
  • crop_width / crop_height (default 512×384) - the fixed crop canvas size. Must match the generation resolution on the H3 side - this is the #1 wiring mistake.
  • smoothing_preset - the seven presets (balanced, stable_max, stable_extreme, cinematic_push, responsive, static_shot, cg_animation) pick the smoothing window and method (Gaussian or Savitzky-Golay) for the track. The tooltips are genuinely useful: stable_extreme for shakycam/low-bitrate footage, responsive for fast handheld moves, static_shot for a locked tripod, cg_animation for clean renders that need almost no smoothing. balanced really does handle ~80% of footage.

Outputs: crop_img (IMAGE), transform (H3FACEXFORM), track_data (SAM3_TRACK_DATA, the raw track for other SAM3 nodes), masks (MASK - the per-frame object masks).

How it works

It runs ComfyUI's native SAM3_VideoTrack + SAM3_TrackToMask, derives a bounding box per frame from the masks, then interpolates and smooths the box centers and sizes over time (Gaussian or Savitzky-Golay per the preset) so the crop doesn't jitter. From the smoothed boxes it builds an affine transform per frame - that's the transform you pass around - and crops everything to the fixed canvas. Unstable tracks get their confidence recorded as per-frame weights, which downstream nodes use to fade out bad frames instead of glitching.

Common failure modes

The big one: "SAM3 did not detect the requested object in any frame. Lower detection_threshold or change pos." - that's the node telling you the description didn't match anything, or the threshold was too strict. Also note this uses ComfyUI's native SAM3 (comfy_extras.nodes_sam3), not the third-party ComfyUI-SAM3 pack - which in 2026 had a genuinely bad reputation for installing its own dependencies and breaking other node packs. Don't install that one for this; you only need the native support plus the checkpoint.

Installing it

From cardenluo/ComfyUI-Apt_Preset:

cd ComfyUI/custom_nodes
git clone https://github.com/cardenluo/ComfyUI-Apt_Preset
pip install -r requirements.txt   # or install.bat

Then drop sam3.1_multiplex_fp16.safetensors into ComfyUI/models/checkpoints and restart. You need a ComfyUI build recent enough to ship native SAM3 nodes.

CategoryApt_Preset/AD

Inputs (10)

NameTypeDefaultDescription
imageIMAGE
detection_thresholdFLOAT0.500–1
max_objectsINT40–64
detect_intervalINT11–10000
ckpt_nameCOMBOsam3.1_multiplex_fp16.safetensors0 options:
posSTRING
crop_factorFLOAT3.01–8
crop_widthINT512128–1344
crop_heightINT384128–1344
smoothing_presetCOMBObalanced balanced : 通用平衡档,适合 80% 的素材。 stable_max : 强抗抖,适合三脚架/稳定器/采访镜头。 stable_extreme : 极端抗抖,适合夜景、低码率、720p 以下、老手机这类检测框抖动严重的素材。 cinematic_push : 保留推镜/拉镜节奏,适合广告、MV、电影感镜头(savgol 中窗口)。 responsive : 高灵敏度跟随,适合手持快速转头、快速运镜、动作幅度大的素材。 static_shot : 锁死三脚架/产品照,size 平滑窗口开到最大(151 帧),轨迹最稳。 cg_animation : 最小平滑,用于 CG 动画 / 游戏录屏这类本身无像素噪声、检测框极稳的渲染素材。

Outputs (4)

NameTypeDescription
crop_imgIMAGE
transformH3FACEXFORM
track_dataSAM3_TRACK_DATA
masksMASK