SeC Video Segmentation

You have a video, you want a clean mask that follows one object through the whole thing - for inpainting it out, recoloring it, compositing it onto a new background, driving a video effect. That's what this node does. You point at the thing on one frame, and it hands you back a mask for every frame with that object tracked through movement, occlusions, and scene cuts.

If you've done image masking with SAM or SAM 2, this is the video-native cousin, but with a twist that actually matters. SAM 2 tracks by visual similarity - it remembers what the pixels looked like and hunts for the matching blob in the next frame. SeC ("Segment Concept") runs the object through a vision-language model first and builds an actual concept of what it is, then tracks that. The practical upshot: when your subject turns around, gets half-hidden behind something, or the shot cuts to a different angle, feature-matching trackers lose the plot and SeC mostly doesn't. Its own benchmark puts it +11.8 points over SAM 2.1 on the hard cases. Fair warning: it's a new and fairly niche model, so you won't find a mountain of community workflows yet - but the mechanism is real, not marketing.

How it works

You give it the video as an IMAGE batch and at least one visual prompt on a single frame. SeC's LVLM studies that object, keeps a "keyframe bank" of diverse views of it, and blends semantic reasoning with plain feature matching depending on how confusing the scene gets. The output always matches your input frame count - 100 frames in, 100 masks out.

The inputs that matter

Two are required: model (from the SeC Model Loader) and frames (your video). Then you need at least one prompt - points, a bbox, or a mask:

• positive_points / negative_points - JSON like [{"x": 100, "y": 200}]. Positives say "this is the thing," negatives say "not that." The simplest way in.
• bbox - a box around the object. Combine with points for two-stage refinement: box establishes the region, points pick out what inside it you actually want (box a person, point on the shirt, get just the shirt).
• input_mask - feed a mask you already have and let SeC track it.
• annotation_frame_idx - which frame your prompt is on. Defaults to 0, but point at the frame where the object is clearest, not necessarily the first.
• tracking_direction - forward (frame N to end), backward (frame N to start), or bidirectional (both ways). Frames outside the tracked range come back blank, so if your object is best in the middle, annotate there and go bidirectional.
• object_id - bump this to track multiple objects across separate runs.

Outputs are masks (MASK) and object_ids (INT). The masks wire straight into a video inpaint, a compositing chain, or a "grow mask + feather" cleanup before you use them.

Two knobs worth knowing but leaving alone at first: offload_video_to_cpu (your main VRAM lever, ~3% slower) and mllm_memory_size (how many keyframes the LVLM reasons over - default 12, the paper used 7, more helps complex scenes with basically no VRAM cost).

How to install it

• ComfyUI Manager: search "SeC" or "SecNodes", install, restart.
• Manual: cd ComfyUI/custom_nodes && git clone https://github.com/9nate-drake/Comfyui-SecNodes, then pip install -r requirements.txt from inside the folder, and restart.

You also need the model. Grab SeC-4B-fp16.safetensors (7.35 GB) from VeryAladeen/Sec-4B and drop it in ComfyUI/models/sams/. Budget 10 GB VRAM minimum (with video offloading on), 16 GB+ to run comfortably.

Common issues

CUDA out of memory - in order: turn on offload_video_to_cpu, drop mllm_memory_size to 5–10, then split the clip into smaller batches. The README's confirmed floor is 10 GB with FP16/BF16.

Blank frames in your output - that's not a bug, that's tracking direction. Forward from frame 50 leaves 0–49 empty. Annotate at the right frame or go bidirectional.

Wobbly, unstable tracking - if you gave it only a mask or only a bbox, that's the known weak spot; both the mask and bbox paths are less stable alone. Add a couple of coordinate points to anchor it.

Negative points doing nothing (or weird things) - with a mask, keep negatives near the masked region. Points more than ~50px outside the mask get ignored or misbehave, and you'll see a console warning.

Running several clips back to back - set auto_unload_model to false so it doesn't reload the 7 GB model between every run.