Florence2 Coordinates

This is the little adapter that makes text-driven masking work. On its own it doesn't detect or segment anything - it's glue. Florence-2 (Microsoft's vision-language model) looks at your image and returns detections as structured JSON: "here's the box for the car, here's the box for the dog." SAM2 wants coordinates or a bounding box, not JSON. Florence2toCoordinates is the piece in the middle that translates one into the other.

Why you'd want this: it's the whole point-and-click step, automated. Instead of hand-placing points on an object, you let Florence-2 find it by name ("the red dress," "the person on the left"), and this node hands the location straight to Sam2Segmentation. Text goes in one end, a precise mask comes out the other. That's the pattern the community converged on for automated masking - a VLM grounds the object, its box becomes the mask prompt, and you never touch the canvas. It's how you build a workflow that masks the same kind of thing across a hundred images without a human clicking each one.

How it works

Florence-2 (run via kijai's companion ComfyUI-Florence2 pack) has a grounding/detection mode that outputs boxes and labels as JSON. This node parses that JSON and pulls out two things you can actually wire into SAM2: the center point of a detection, and its bounding box. Simple in concept; it's just picking the right detection out of the blob and reshaping it.

The gotcha worth knowing up front, from how these detectors behave: grounding boxes are often tighter than you expect - they wrap the cup and not the saucer. If your downstream mask feels cropped, that's usually the detector being literal, not this node misbehaving.

The inputs and outputs that matter

Three required inputs:

• data (JSON) - the raw output from a Florence-2 node. This is the input that matters; everything depends on Florence-2 running in a detection/grounding mode that actually produces boxes.
• index (STRING, default "0") - which detection to grab when Florence-2 found several. 0 is the first one. If Florence-2 returns three boxes and you want the second, this is the knob. Getting nothing useful downstream usually means you're pointing at the wrong index.
• batch (BOOLEAN, default false) - turn on when you're processing a batch of images and want detections handled per-image rather than as one set.

Two outputs, and you pick based on how you're prompting SAM2:

• center_coordinates (STRING) - the detection's center point. Wire this into Sam2Segmentation's coordinates_positive when you want a point prompt.
• bboxes (BBOX) - the bounding box. Wire this into Sam2Segmentation's bboxes input when you want a box prompt. Boxes are generally the more reliable of the two for SAM2 - a box says "the object is in here" and leaves less room for a point to land on the wrong sub-region.

How to install it

It ships in the SAM2 pack. ComfyUI Manager → search ComfyUI-segment-anything-2 → install → restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-segment-anything-2

You'll also want kijai's ComfyUI-Florence2 pack installed to produce the data this node consumes - it's a separate repo. On ComfyICU the SAM2 pack is in the shared image.

Common issues

• No output / empty coordinates. Nine times out of ten the data isn't in the shape this node wants - Florence-2 has to be in a detection/grounding task that emits boxes, not a plain caption task. A caption is text, not a box; there's nothing to convert.
• Grabbing the wrong object. Florence-2 found multiple things and index is pointing at the wrong one. Bump the index, or tighten your Florence-2 prompt so it detects fewer things.
• The mask comes out too tight. That's the detection box being literal, not this node. If SAM2 needs a little more context, prefer feeding the bboxes output over the single center point, or add manual points.