Sam2VideoSegmentation

This is the node that does the actual tracking. You've already told SAM2 what to follow with Sam2VideoSegmentationAddPoints - pointed at your object, tagged it, built up an inference state. This node takes that state and propagates it across the whole clip, spitting out a mask for every frame. Click a subject once, get a mask sequence that follows it through the video. That's the SAM2 video pipeline's payoff, and this is the last step of it.

Why this matters in a real workflow: a per-frame mask sequence is what unlocks video inpainting, background swaps, rotoscoping, object removal, and region-locked effects that stay glued to a moving subject. The KB's short version of SAM 2 is exactly this - "improved accuracy, video support (segment objects across video frames)." Doing that by hand, frame by frame, is the tedium this replaces.

How it works

SAM2's video model carries a memory of what it's tracking (the inference state the AddPoints node built). This node runs propagation: starting from the frames you prompted, it walks forward and backward through the clip, using that memory to re-locate the object in each frame even as it moves, turns, or gets briefly occluded. Out comes one mask per frame, in order. Nothing new is prompted here - it's pure execution of the plan AddPoints set up. If the tracking is wrong, you fix it upstream in AddPoints, not here.

Because it processes the sequence in one pass and holds state the whole way, this is the memory- and time-hungry step of the pack. Clip length is the cost driver.

The inputs and outputs that matter

Three required inputs, and two of them come straight from earlier nodes:

• sam2_model (SAM2MODEL) - from the loader in video mode. Same rule as the rest of the video pipeline; single-image mode won't work.
• inference_state (SAM2INFERENCESTATE) - the payload from Sam2VideoSegmentationAddPoints. This carries every object you prompted. This is the input that makes the node do its job.
• keep_model_loaded (BOOLEAN, default true) - note it defaults to true here, unlike the single-image node. Video runs tend to be iterative, so keeping the model resident between runs saves reload time; the tradeoff is held VRAM.

Output:

• mask (MASK) - the per-frame mask sequence. Feed it into video inpainting, a GrowMask/blur for feathering, an ImageCompositeMasked for background swaps, or straight to a preview to check the tracking.

How to install it

Ships with the pack. ComfyUI Manager → search ComfyUI-segment-anything-2 → install → restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-segment-anything-2

Restart after. The loader handles the model download; ComfyICU has the pack in its shared image.

Common issues

• Loader not in video mode. If the model didn't come from a loader set to video, this node has nothing to propagate. The most common failure across the whole video pipeline.
• The mask is wrong but the node "worked." This node only executes; it doesn't decide what to track. Bad masks are an AddPoints problem - seed on a cleaner frame, add points, add negatives. Re-running this node with the same state won't change anything.
• Out of memory / very slow on long clips. Video segmentation holds state across every frame, so cost scales with length. Trim the clip, or step down to a smaller Hiera variant (small/tiny) in the loader. If you're iterating and VRAM's tight, you can set keep_model_loaded to false to free it between runs at the cost of reload time.
• Tracking wanders on fast motion or occlusion. A known SAM2 limit, not a bug in the node. More prompt points upstream, and seeding the object on the frame where it's clearest, are your levers.