EdgeTAM Video Tracker

The one-line version

You point at a thing in frame one, and this node hands you back a segmentation mask for every frame of the video. That's the whole trick - and it's the hard part of a pile of workflows: rotoscoping a subject out for compositing, feeding a moving mask into a video inpainting pass, or generating the region map for a selective video edit. Doing that by hand, frame by frame, is miserable. This does it in one pass.

What it actually is

EdgeTAMVideoTracker is the payoff half of the lum3on/comfyui_EdgeTAM pack. EdgeTAM is Meta Reality Labs' "On-Device Track Anything Model" (CVPR 2025) - an optimized SAM 2 variant built to run fast on consumer hardware. The README claims roughly 22x faster than SAM 2; treat that as paper framing, but the point stands. This is the SAM 2 memory-propagation approach with a leaner Hiera backbone, and it genuinely runs on a normal GPU.

The pack ships exactly two nodes, and this is the one that does the tracking. Its sibling, InteractiveMaskEditor, gathers the clicks; this node turns them into masks.

How it works

Mechanically it's the classic SAM2 click-to-propagate flow:

1. Your video_frames batch gets written to a temp MP4 on disk (mp4v codec), because EdgeTAM's video predictor expects a video file rather than a tensor batch.
2. Your point prompts - pixel coordinates plus labels (1 = include, 0 = exclude) - are normalized to the 0–1 range and applied to frame 0.
3. The model propagates that segmentation forward through every frame with its memory mechanism, and each per-frame mask comes back as a ComfyUI MASK.

One thing worth knowing: it tracks a single object (internally obj_id 0). Want two objects tracked? Run two tracker nodes.

The inputs that matter

• video_frames (required): the IMAGE batch. Wire it from whatever loads your video.
• mask_data: a JSON string shaped {"points": [[x,y],...], "labels": [1,0,...]}. Normally this is wired from the InteractiveMaskEditor and hidden. Leave it disconnected and the node silently uses a default point at (100,100) - that trap has wasted a few evenings.
• device: auto/cuda/cpu/mps. Leave it on auto; it picks CUDA when it can, and the pack even monkey-patches a bfloat16 problem on older macOS.
• max_frames: cap how many frames you process (0 = all, hard ceiling 1000). Set it on long clips if you're memory-bound.
• overlay_masks: draws the green overlay (default on).

The outputs

Three of them, and you mostly care about masks (MASK) - a batch of per-frame binary masks you can wire into a video combine (the bundled example workflow uses VHS_VideoCombine), an inpaint, or a composite. tracked_frames is just your original frames echoed back, and overlay_frames is the frames with the mask painted on at 60% green, handy for a quick preview.

Installing it

ComfyUI Manager (search "EdgeTAM"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/lum3on/comfyui_EdgeTAM

Then restart. Brace yourself: the first run is not a fast run. On import the pack checks for sam2, and if it's missing it clones the whole facebookresearch/EdgeTAM repo, pip install -e .s it, and downloads the edgetam.pt checkpoint. That's hydra-core, omegaconf, moviepy, opencv and friends landing in your environment at once - the classic "why is my first run hanging" moment from the ecosystem docs. Requirements: Python 3.10+, PyTorch 2.3.1+.

Troubleshooting

• Empty or black output with no error. The tracker catches exceptions and returns zero tensors instead of raising. The real error lives in the console - read it.
• Wrong thing tracked. That's the (100,100) fallback biting when mask_data is empty. Always run through the editor or feed real JSON.
• Drifting mid-clip. EdgeTAM is appearance-driven. If your subject is occluded or changes look, the mask wanders - that's inherent to this approach, not a setting you missed. More include points on the object plus an exclude point on the background helps.
• Slow start. Model load plus the temp video encode happen once per run. Give it a minute.

Honest take: this pack is early and quiet - essentially unmentioned in the community, and its USAGE.md describes extra editor nodes the shipped code doesn't actually contain. You're adopting a wrapper around a solid model, not a polished ecosystem. If you just need a moving mask and can handle rough edges, it works. If you want batteries included, the more established SAM2-based ComfyUI ports (or a concept-aware model like SeC-4B) are where the community actually hangs out.