Detect Region UV + Build Metadata Image

This node has a slightly weird pitch, and it's worth getting straight before you wire it up. Most detection nodes in ComfyUI hand you a mask you then feed into an inpaint or a detailer - the Impact Pack SEGS loop, FaceDetailer, all that (the KB calls it "detect, crop, re-render, paste back"). Detect Region UV + Build Metadata Image does the detection but stops one step short: instead of a mask, it hands you a normalized polygon for every object you name, and it stuffs those polygons into the image's metadata so a downstream tool can pick them up later. Think of it as "name the region, get its outline as data."

Why UV? Because the polygons are stored as UV vertices - normalized (u,v) coordinates in the 0..1 range rather than pixel positions. A box drawn at [[0.12, 0.18], [0.45, 0.17], [0.48, 0.79], [0.11, 0.81]] means the same thing at 512×512 and at 4096×4096. That makes the metadata resolution-independent, which is the whole point of the format: it travels with the file and reconstructs on whatever canvas you later drop it onto.

How it works

The mechanism is textbook Grounded SAM, wrapped so you never touch the parts. Grounding DINO runs first - give it text targets and it returns bounding boxes with confidence scores. Then SAM 2.1 (the hiera-small checkpoint) takes each box and segments it into a precise mask. Finally the node's polygon extractor traces the mask outline, simplifies it, and filters out anything smaller than min_area_ratio.

Two model files get pulled automatically on your first run (more on that below), and the author wired a progress readout into the node so the download isn't a mystery.

The inputs that actually matter

You'll fiddle with maybe four of these:

• targets_json - the text targets, default person, car. This is friendlier than it looks: comma-separated, Chinese commas, 、, semicolons, newlines, or a proper JSON array like ["person", "dog", "car"] all parse fine.
• box_threshold (0.35) and text_threshold (0.25) - Grounding DINO's detection and text-match thresholds. Higher = stricter, fewer boxes. If it's not finding anything, drop box_threshold; if it's finding junk, raise it.
• polygon_simplify (0.005) - how aggressively the outline is smoothed. Crank it up when a silhouette comes back with a thousand points.
• min_area_ratio (0.001) - drops tiny specks of mask by fraction of total image area. The classic "it detected a button on the shirt" filter.

max_detections_per_target (default 3) caps how many regions each label keeps. metadata_field sets the key it writes under - default region_uv_vertices. And line_thickness / font_scale only affect the preview drawing, not the data.

Outputs

Three of them:

• metadata_image - the custom REGION_UV_METADATA_IMAGE type. It only plugs into this pack's Save Metadata Image As PNG/WEBP node; that's deliberate, not a bug.
• preview_image - the source image with boxes, labels, and confidence drawn on, so you can see what got caught.
• metadata_json - the payload as a string. Handy for sanity-checking or for wiring into something that consumes raw JSON.

The payload is a dict of target → list of {score, polygon_uv} entries; on a batch, results are grouped by batch_index.

Install

ComfyUI Manager (search "comfyui_region_uv_metadata"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/csonxx/comfyui_region_uv_metadata
pip install -r ComfyUI/custom_nodes/comfyui_region_uv_metadata/requirements.txt

then restart. Dependencies are numpy, opencv-python, Pillow, torch, and transformers>=4.57.1 - the transformers floor matters because that's where the SAM2 integration lives. First run downloads IDEA-Research/grounding-dino-tiny and facebook/sam2.1-hiera-small into the node's models/ folder (progress shows in the node). If you'd rather not wait, pre-download those two HF repos into custom_nodes/comfyui_region_uv_metadata/models/grounding-dino-tiny and .../sam2.1-hiera-small, or point COMFY_REGION_GROUNDING_MODEL / COMFY_REGION_SAM2_MODEL at your own paths.

Troubleshooting

• Nothing detected → drop box_threshold toward 0.25, make sure the target word matches what's actually in the image.
• Detects too much junk → raise box_threshold and min_area_ratio.
• Ridiculous polygon shapes → raise polygon_simplify a notch; if you go too far you'll get a lumpy potato instead of an outline.
• preview_image looks right but the saved file has no metadata → you hit the metadata toggle: ComfyUI's "Disable saving prompt metadata in files" (or --disable-metadata) stops the save node from writing anything. Flip that off.
• First run is genuinely slow - it's loading two transformers models. Subsequent runs are fast because the runtime caches them.

One honest caveat: the README is Chinese-first and this is a small pack, so don't expect a lively support thread. But the README is unusually thorough, and the source is clean enough that the failure modes are the ones above.