Nodes/ComfyUI-CustomNodePacks/Mask Placement — Prompt/Ref → Place → Track
ComfyUI Node

Mask Placement — Prompt/Ref → Place → Track

Drop a cutout anywhere in the frame and carry it through the video

By Code2Collapse·Created 6 months ago·Updated a day ago· 52
Mask Placement — Prompt/Ref → Place → Track
  • images
  • source_mask
  • source_image
  • ref_image
  • sam_model
  • masks
  • placed_rgb
  • overlay_preview
  • info
prompt
grounding_modelgroundingdino_swinb_cogcoor
track_modestatic
anchor_frame0
placement_json
feather_px6

The use case: you want to take a cutout - your dog, a logo, a car - and drop it onto a different video, positioned and warped like it belongs there, with a clean alpha. Mask Placement (MEC) is that node. It sources an alpha three ways, lets you position it with a draggable four-corner perspective quad, and propagates the placement across every frame of the clip.

The honest caveat up front: this is "Slice 1" of a planned feature. Static placement is live and fully usable - great for stills and locked-off shots. The fancier object_track (Cutie) and landmark_lock modes are staged, and the node itself tells you in info when it fell back to static instead of pretending.

How you feed it an alpha

Three source modes:

  1. Wire a source_mask - any alpha from SAM, the matting nodes, whatever. This skips segmentation entirely. Optionally add source_image (the RGB the mask cuts out of) so the placed output carries actual pixels.
  2. Give it a prompt ("dog", "car") plus a wired sam_model and a grounding_model (GroundingDINO). It segments the object for you, using either the video's anchor_frame or a separate ref_image (a photo of the dog from another angle).
  3. Nothing - it places an empty quad; you're really only doing this to build the placement.

Then the editor widget (the placement_json, in anchor-frame pixel coords, corners as TL/TR/BR/BL) gives you the perspective quad - position, scale, rotate, warp. Empty JSON means a centered quad at 60% size, so the node always produces something sane. feather_px (default 6) softens the placed alpha's edge.

Outputs

  • masks - per-frame placed alpha; feed to inpaint/composite.
  • placed_rgb - the source pixels over black, per frame, for compositing.
  • overlay_preview - frames with the placement tinted and the quad drawn; this is how you verify the shot.
  • info - JSON with the source mode, the quad, per-frame status, and which tracking actually ran (vs. silently fell back to static).

Where it fits and what to expect

For a locked-off plate it's a genuinely pleasant way to comp a prop or an actor's insert: cut → place → composite → done, all in one node. For a moving shot, static placement means the cutout stays glued to its quad while the camera moves - which is wrong. That's the boundary of Slice 1, and the node says so. Don't expect motion-tracking from it yet; use the pack's MaskTracker or SAM2 video propagation when the plate moves.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/Code2Collapse/ComfyUI-CustomNodePacks.git
pip install opencv-python>=4.7.0 scipy>=1.10.0

or ComfyUI Manager → search "CustomNodePacks". Prompt-based segmentation needs a SAM checkpoint in ComfyUI/models/sams/ or models/sam2/ and a GroundingDINO model, so expect those downloads. If your placed_rgb comes back black, you forgot source_image. If info says everything fell back to static, you've hit the Slice-1 boundary - read the info string, it's honest about it.

CategoryMEC/Masking

Inputs (11)

NameTypeDefaultDescription
imagesIMAGEVideo frames or a single still (B,H,W,3).
promptSTRINGWhat to cut out, e.g. 'dog', 'car', 'mouth'. Needs sam_model (+ a GroundingDINO model) wired. Ignored when source_mask is wired.
grounding_modelCOMBOgroundingdino_swinb_cogcoorGroundingDINO model for text->box grounding (same list as SAM Mask Generator).
track_modeCOMBOstaticHow the placement follows the video. Slice 1: 'static' is live; 'object_track' (Cutie) and 'landmark_lock' fall back to static for now and say so in `info`.
anchor_frameINT00–99999The frame the quad was placed on (and segmented from, when using prompt without ref_image).
placement_jsonSTRINGEditor-owned: {"corners":[[x,y]x4 TL,TR,BR,BL], "feather":px} in anchor-frame pixel coords. Empty = centered 60%-size quad.
feather_pxINT60–128Edge feather (Gaussian, px) applied to the placed alpha. placement_json's feather wins when present.
source_maskoptMASKBYO alpha (e.g. SAM Mask Generator / matting output). Skips prompt segmentation entirely.
source_imageoptIMAGERGB that source_mask cuts out of (for the placed_rgb output). When absent, the mask is placed without pixels.
ref_imageoptIMAGESegment the prompt from THIS image instead of the video's anchor frame (e.g. a photo of the dog you want).
sam_modeloptSAM_MODELFrom SAM Model Loader — needed only for prompt-based segmentation.

Outputs (4)

NameTypeDescription
masksMASKPer-frame placed alpha (B,H,W float 0..1). Feed to inpaint/composite.
placed_rgbIMAGEPer-frame RGB of the placed source pixels over black (use with `masks` to composite).
overlay_previewIMAGEFrames with the placement tinted + quad drawn — visual verification.
infoSTRINGJSON: source mode, quad, per-frame status, and which tracking actually ran.