Interactive Mask Editor

The one-line version

Wire in your video frames, run the workflow, and ComfyUI pauses while a full-screen editor pops up over the first frame. You click the thing you want tracked, hit Preview Mask to see the segmentation, save - and the workflow resumes, handing your clicks to the tracker. No coordinates, no JSON, no guesswork. That's the entire job of this node: it's the human input stage for the EdgeTAMVideoTracker.

Why you need it

The tracker needs point prompts - a list of [x,y] coordinates and labels (1 = include, 0 = exclude) - before it can track anything. You could type those numbers by hand, but that's how you end up clicking (240, 137) on the wrong object and wondering why nothing matches. This node is the friendly front end: you click directly on the actual first frame, see what the model thinks you meant, and only then hand it over.

How it works

It's a deliberately simple server/frontend dance:

1. On execute it grabs image[0] (the first frame of your video batch), base64-encodes it, and pushes a websocket event to the browser UI with a session id.
2. The frontend renders a full-screen editor - green dots for include points (left-click), red for exclude (right-click).
3. Preview Mask sends your current points back to the server, which runs a singleton SAM2 image predictor and returns a live mask overlay. This is the feature that makes the node usable: you see the actual segmentation before committing.
4. Save and Continue posts the points back, the node's worker thread unblocks, and the clicks come back as a JSON string. Cancel and Stop Workflow aborts the whole queue.

Because the node blocks its worker thread on a threading.Event while you're clicking, the entire queue pauses until you save. That's the intended UX - but it also means this node needs a human in the room. You can't leave it in an unattended batch run.

The automation escape hatch

The same node works headless. Connect optional_mask_data with a JSON string and the editor is skipped entirely - the string passes straight through:

{"points": [[320, 240], [400, 300]], "labels": [1, 0]}

That's the batch-processing path the README advertises: define points once in a string node, rerun over many videos, never see the editor. For a first run, though, do it interactive. The preview is worth one manual pass.

Inputs and output

• image (required): your video frame batch - the same one you're feeding the tracker.
• optional_mask_data: the automation JSON above. Leave it disconnected for interactive mode.
• Output mask_data (STRING): a JSON blob with points, labels, width and height. Wire it straight into the EdgeTAMVideoTracker's mask_data input. That's the whole connection: editor → tracker.

Installing it and the gotchas

Install is identical to its sibling - ComfyUI Manager (search "EdgeTAM") or:

cd ComfyUI/custom_nodes
git clone https://github.com/lum3on/comfyui_EdgeTAM

Then restart. The first run triggers a heavy auto-install: the pack clones the facebookresearch/EdgeTAM repo, pip-installs it, and pulls down the edgetam.pt checkpoint, along with hydra-core, omegaconf, moviepy and friends. Python 3.10+, PyTorch 2.3.1+. Budget a patient first launch.

Where people get burned:

• Close the editor without saving and you get an empty point list back - and the tracker's fallback then clicks (100,100) for you. Not what you wanted. Use Cancel if you changed your mind; it aborts cleanly.
• The first Preview Mask is slow because it lazily loads the SAM2 image predictor on the server side. Later previews are quick.
• It only ever prompts frame one. If your subject is hidden in the first frame, this node can't help - there's no re-prompting mid-clip.
• Full-screen overlay on a small monitor is cramped. It's built to be unmissable, not pretty.

Straight talk: this is the more polished half of a small, quiet pack (the author shipped it mid-2025, and the community hasn't exactly discovered it yet). It's a thin, honest wrapper around Meta's EdgeTAM, and the interactive flow - click, preview, save - is genuinely nicer than hand-typing SAM2 point arrays. Just budget for the first-run install and accept that the node only works when you're sitting there to answer it.