SAM3 Video Segmentation

This is step one of SAM3's killer feature: tracking an object through a video. SAM3 doesn't just mask a still - it can follow the same thing across every frame, which is what you need for rotoscoping, object removal, or masking a subject for video inpainting. But it needs to know what to track first, and that's this node's job. You give it your frames and tell it, on one reference frame, which object you mean - by text, by clicking points, or by drawing a box. It sets up the tracking state; SAM3 Propagate does the actual frame-to-frame chase.

This mirrors how the whole SAM lineage handles video: seed the target on a frame, then let the model carry that identity forward and backward. SAM3 makes the seeding part flexible - the same node handles text, point, or box prompts through one prompt_mode switch. It's the current top tier for this; Comfy Org even pointed people at SAM3 to generate the input masks for video object-removal pipelines.

How it works

The node ingests your video as a batch of frames and builds a SAM3_VIDEO_STATE - an object that holds the frames plus your seed prompt on the chosen frame. It doesn't propagate anything yet. It just records "on frame N, this is the object I care about." That state is the baton you hand to SAM3 Propagate, which walks the object across the timeline.

The inputs and outputs that matter

• video_frames (IMAGE) - your video as a frame batch. Load it however you normally load video into ComfyUI.
• prompt_mode (default text; also point, box) - how you're pointing at the target. Pick the one matching the prompt you're feeding in.
• frame_idx (default 0) - which frame you're marking the object on. Usually the first, but pick whichever frame shows your subject clearly.
• score_threshold (default 0.3) - the detection confidence bar for accepting the target.

Then the prompt itself, matched to prompt_mode: text_prompt (a noun like person), or positive_points / negative_points (SAM3_POINTS_PROMPT), or positive_boxes / negative_boxes (SAM3_BOXES_PROMPT). Feed the one your mode expects.

Output is a single video_state (SAM3_VIDEO_STATE) - wire it straight into SAM3 Propagate. On its own it produces no masks; it's the setup, not the payoff.

How to install it

Ships with the pack. Manager: search SAM3, install the highest version. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/PozzettiAndrea/ComfyUI-SAM3.git
cd ComfyUI-SAM3
pip install -r requirements.txt --upgrade
python install.py

then restart. sam3.pt belongs in ComfyUI/models/sam3/, fetched by install.py. The README warns about an experimental comfy-env/pixi one-click install. Video work is heavier than stills - a whole frame batch lives in memory - so give this room.

Common issues & troubleshooting

It seeds the wrong object or nothing. The prompt has to match prompt_mode. If you're in text mode, fill text_prompt; if point, feed positive_points; if box, feed positive_boxes. Mismatched mode and prompt is the classic first mistake. Also make sure frame_idx points at a frame where the target is actually visible and clear.

Nothing tracks. Right - this node only sets up the state. Masks come out of SAM3 Propagate, which consumes the video_state. If your graph ends here, you've only done half the job.

Detection misses on the seed frame. Lower score_threshold, or switch to a point/box prompt for a crisper hint than text can give on a busy frame.

Runs out of memory on long clips. Video segmentation holds the frame batch plus model state. Trim the clip, shrink resolution, or process in chunks - SAM3 video is genuinely memory-hungry.