MatAnyone2 Interactive SAM

This is the node you'll actually spend your time in. MatAnyoneInteractiveSAM is the whole reason the MatAnyone2 Video Matting pack exists: a point-and-click SAM editor running inside ComfyUI - no external app, no API key, no exporting frames to Photoshop to draw a mask. You click the thing you want cut, and it hands the mask to the Matte node to propagate.

The workflow is deliberately two-pass, and it trips everyone up the first time:

1. Queue once with no points set - the node saves the frame to ComfyUI's temp folder, opens the editor, and returns a blank mask. You'll see the editor UI pop over the node.
2. Left-click the foreground, right-click the background, per the README. Add as many points as it takes.
3. Hit Apply and queue again. Now the editor state has real points, so SAM loads and the node returns the merged mask.

How the mechanism works

The editor isn't a gimmick floating over the canvas. The pack registers real backend routes on ComfyUI's PromptServer - /matanyone2/interactive/create_session, /predict, and /close_session. When the editor opens, it creates a SAM session (predictor loaded, image embedded), and every click hits /predict, which runs SAM against your points and streams back a live preview as a data URL. The session is cached server-side with a 15-minute TTL, so scrubbing points feels responsive instead of re-encoding everything each time.

The genuinely nice part is multi-target masking. Instead of one blob of points, you can build several named targets (Mask 1, Mask 2, …), each with its own points and its own mask_choice. Each target gets its own SAM prediction, and the node merges them with a simple max-then-threshold - merge_threshold defaults to 0.5. That's how you cut "the person and the chair they're holding" as one clean matte instead of fighting SAM for a single union. The preview overlay opacity (preview_opacity, default 0.65) is just for your eyes.

The whole editor state lives in the editor_state STRING output. That's JSON - you can copy it, save it, or paste it back into the editor_state input on a fresh node to reproduce the exact same edit without re-clicking.

The inputs and outputs

Required inputs: image (the single frame from Select Frame), sam_model_type (vit_h default, or vit_l/vit_b), checkpoint_path (blank auto-downloads to ComfyUI/models/sams/), device (auto/cuda/mps/cpu), plus the editor-state/preview knobs above. Outputs: mask (MASK - the merged first-frame mask, which feeds MatAnyone Matte), preview (IMAGE overlay), editor_state, and target_count (INT, so you know how many mask targets you built).

Install & the gotchas

Same as the whole pack - ComfyUI Manager search "MatAnyone2", or git clone https://github.com/dreamrec/MatAnyone2_ComfyUI into custom_nodes, pip install -r requirements.txt, python install.py, restart. SAM ViT-H (~2.5 GB) auto-downloads on first use; it's a big file, so the first editor open has a wait.

Real-world traps, straight from the README:

• "The editor doesn't open" - queue the workflow once so the first-frame preview exists. The editor needs that first pass.
• "It works in the first queue but returns a black mask" - you queued with no points. Set your points and Apply, then queue.
• VRAM tight? Drop sam_model_type to vit_b - it's dramatically lighter than vit_h and fine for a first-frame mask, which is all you need here since MatAnyone does the heavy propagation.

One honest warning: the editor is doing real SAM inference on every click, so on a weak GPU the live preview lags. It's not a bug - you're watching a 2.5 GB model think. Click deliberately and it's fine.