SAM2Matting Video

Here's the pitch that separates this from every per-frame background-remover you've tried: you paint a rough mask on one frame, and it tracks that subject through the entire clip, forward and backward, with soft edges that stay temporally consistent. No keyframing every fifth frame, no flickering cutout. It's SAM 2's streaming memory doing the heavy lifting - the same machinery that tracks objects through video - pointed at a foreground-matting task instead of segmentation.

How it works

SAM2Matting Video treats your ordered IMAGE batch as one video clip. The initial_mask on the chosen mask_frame is binarized with mask_threshold into a tracking prompt, then the predictor propagates a single temporal state: forward from the seed frame, and backward too when mask_frame isn't zero. Because it's one continuous state rather than independent frame runs, the matte doesn't jump around - the model remembers what the subject looks like as it moves, turns, and briefly occludes.

The inputs that matter

• initial_mask - your one-frame seed. White is foreground, black is background. It doesn't need to be clean; rough is fine.
• mask_frame - the zero-based frame that matches your mask. Get this wrong and you're seeding the wrong frame, which is the most common cause of a garbage result.
• mask_threshold - only turns the seed into a binary prompt. It does not threshold the soft output matte, so the edges stay soft regardless.
• memory_mode - balanced (default) is the sensible middle. low_vram offloads frames and temporal state to CPU; maximum_speed keeps everything on the model device and uses the most VRAM.

The output and where it goes

It returns one thing: alpha, a soft foreground-opacity MASK for every frame. Use this node when that alpha has to feed other ComfyUI image nodes - compositing, further mask processing, that kind of thing. For transparent video, connect your original video batch to Join Image with Alpha, but watch the trap: ComfyUI's Join Image with Alpha uses inverse MASK semantics, so you must pass alpha through Invert Mask first, exactly as the pack's example workflow does. Get that backwards and you get a hole where your subject should be.

The node deliberately returns only alpha - no RGB copy, no preview - because ComfyUI caches every returned tensor, even unconnected outputs. Cutting those two outputs drops what it caches from seven float channels per pixel to one, which is a meaningful difference on a long clip.

Where people get burned

The whole decoded frame batch lives in RAM, and it adds up fast: one float32 RGB frame is roughly 10.5 MiB at 720p, 23.7 MiB at 1080p, before mattes. A 500-frame clip is a gigabyte-plus before anything else in your workflow. That's why the pack's streaming node exists for long renders. Keep this tensor node for when you need the alpha inside ComfyUI's image graph, and don't run it on enormous clips on a tight-RAM box - low_vram helps, but only so much. One tracked object per run, seeded by a mask; points, boxes, and multi-object propagation aren't exposed yet, so this is strictly one-subject matting for now.