Mask to Coordinates (SAM2)

Segment Anything is a click-driven model: you give SAM a point (or a few) and it segments whatever object those points belong to. That's great when you're clicking interactively, and a pain when you want to segment the same shape on a hundred frames or batch-run a mask you already painted. Mask to Coordinates (class MaskToSAMCoords) is the bridge - it takes a mask you've drawn in ComfyUI's Mask Editor and converts it into the list of coordinates SAM needs, as a JSON string.

So the pipeline becomes: paint the mask once, drop in this node, and out pops [{"x": 512, "y": 340}, ...] - a ready-made point prompt you can feed to a SAM2 node that accepts coordinate JSON. No interactive clicking, no manual coordinate entry. If you're doing video work with SAM2's memory feature, being able to generate those points from a mask per frame is a small superpower.

How it actually works

Mechanically it's a classic image-processing chain, and it's worth knowing because it explains the knobs:

1. The mask tensor is normalized to 0–1, then thresholded: pixels above threshold (default 0.65) become the binary region.
2. OpenCV finds contours in that binary mask, capped at max_regions (default 50) - so a mask with fifty separate blobs gives you up to fifty regions.
3. For each region, it drops points_per_region random points (default 1) inside the contour, using pointPolygonTest to make sure the point actually lands in the shape.

The inputs are exactly those three knobs plus the mask itself. threshold is the one you'll fiddle with most - if your mask is soft-edged or you keep getting the whole canvas, raise it. And note the randomness: each run re-picks points inside the contours, so the exact coordinates change run to run even with the same mask. If you need reproducibility, that's a real quirk to know about.

Output: coordinates, a JSON string. The format [{"x": ..., "y": ...}] matches what SAM2 point-prompt nodes in the ComfyUI ecosystem expect, so wire it straight into one.

Where people get burned

Two failure modes, both handled but worth knowing. First, a solid-color mask (all black or all white) - the node can't find any variation to threshold, so it logs a warning and returns a default point at (0, 0). That will segment the corner of your image, not the thing you wanted. Second, a mask with nothing above the threshold - same deal, default point. Both cases are silent-ish (the warning goes to the console), so if your SAM mask comes out weird, check that your painted mask has actual value range before blaming the model.

Also worth noting: this node does not run SAM. It generates the condition input for a SAM2 node - you still need a SAM2 loader elsewhere in your graph (the model, not this pack, is the heavy part). The node itself is pure NumPy and OpenCV, so it's instant and costs no VRAM.

Installing it

Part of lhaoyun6/ComfyUI-lhyNodes. Install via ComfyUI Manager by searching for lhyNodes, or:

cd ComfyUI/custom_nodes
git clone https://github.com/lihaoyun6/ComfyUI-lhyNodes.git
python -m pip install -r ComfyUI-lhyNodes/requirements.txt

Restart after. The pack's shared requirements are ultralytics, opencv-python, numpy, and yarl - OpenCV is the one doing the heavy lifting here, and it comes with the pack install. No model downloads.

If you're already building SAM2 pipelines, this node saves you from the most tedious part of them. If you're not, it's a neat on-ramp: paint a mask, get click points, feed them to SAM, and suddenly "automatic segmentation of the thing I drew" is a two-node chain.