Sam2RealtimeSegmentation

This is the node that makes the pack worth installing. Feed it a batch of frames - the output of a video decoder, an AnimateDiff sequence, whatever - click one point on the object you care about, and it hands back a clean binary mask for every frame, plus a colored overlay so you can eyeball whether the tracking held. Where image-only SAM2 nodes make you re-click per still, this one runs SAM 2's camera predictor: the first frame seeds the object, and every frame after that is tracked, not re-segmented from scratch. That's the difference between "masking a video" and "masking a video one painful frame at a time."

If you've only met SAM2 through a background-removal pack, this is a different tool for a different job. It's not "cut out the subject of this one image" - it's temporal, and that makes it the right layer under video inpainting, region-specific re-generation across a shot, or isolating an object so you can swap or composite it. Got a 5-second Wan or LTX clip where the character's face needs fixing? This is how you build the consistent mask that survives the motion.

How it works

Under the hood it's the SAM2 video-predictor lifecycle. Frame 0 goes through load_first_frame, then your click points are added as a prompt via add_new_prompt. Every subsequent frame goes through track(), which carries the object's memory across the sequence. The subtle bit: that state lives in the node instance and persists between runs. Run the node again without touching anything and it doesn't restart - it keeps tracking from where it left off, like a camera that never stopped recording. reset_tracking flips it back to a fresh start, which is your "click again on a new object" button.

The inputs that matter

• images (IMAGE) - your frames as a batch, in temporal order. The node assumes frame 0 is the prompt frame, so if your batch is shuffled, your mask follows the wrong object.
• sam2_model (SAM2MODEL) - the handle from DownloadAndLoadSAM2RealtimeModel. They're a matched pair.
• coordinates_positive / coordinates_negative (STRING) - the clicks. This is the top trap for beginners: these are strings, not point widgets, and they're parsed with ast.literal_eval, so the format is a Python list of [x, y] pairs like [[384, 384]]. Positive points say "the object is here," negative points say "not here" - one negative click on an adjacent object is how you stop the tracker glomming onto the wrong thing.
• reset_tracking (BOOLEAN, default false) - set it true to wipe the tracker state and treat the next batch as a new scene with a new prompt.

Outputs

Two. PROCESSED_IMAGES (IMAGE) is the overlay - frame at 70% plus the mask at 30%, so you can watch the tracking in a Preview node. MASK (MASK) is the binary mask per frame, and it's the output you actually build on: wire it into a video inpainting setup, a mask-to-image compositor, or anything that needs per-frame region control.

Common issues

• New clicks don't take effect. You changed coordinates_positive but the node just keeps tracking the old object. That's the persistent state, not a bug - toggle reset_tracking (or use a bypass then enable to force re-init) and the new prompt applies.
• Format errors on coordinates. Missing quotes, wrong nesting, or a stray character makes literal_eval fail and the node silently runs with no prompt at all. The example workflow uses [[384, 384]]; copy that shape exactly.
• It's CUDA-only in practice. The segmentation code hardcodes torch.device("cuda") and fp16 autocast, so the loader's cpu/mps options won't save you here. This node wants an NVIDIA GPU.
• Install can fail before you ever see it. The pack compiles a CUDA extension during pip install and hardcodes an architecture list that excludes Blackwell (RTX 50-series), so budget for a build step - see the loader's article for the full install walkthrough.

It's niche and fiddly in exactly the ways a single-author realtime pack usually is. But if your problem is "keep a mask glued to a moving object," there aren't many ComfyUI-native paths to it, and this one does the thing.