VLM Detections to Bounding Boxes

ComfyUI has a native BOUNDING_BOXES type, and a lot of core and community nodes want it: inpaint crops, region-conditioned generators, SAM-style region tasks. But gokayfem's VLM_nodes pack speaks its own typed dialect - VLM_DETECTIONS - where coordinates live in source-image pixels and every box carries a label and score. VLM Detections to Bounding Boxes is the translator between the two worlds.

It's a small node with a specific job: you detected things, you want the boxes in the format the rest of ComfyUI expects. Wire a VLM_DETECTIONS socket in, get BOUNDING_BOXES out, and the metadata that tells you which box is which.

How it works

Detection records in this pack are [x1, y1, x2, y2] with an exclusive right/bottom edge, in source-image pixels - the same convention core ComfyUI uses, which is why the conversion is lossless rather than approximate. The node flattens the per-frame detection structure into a core BOUNDING_BOXES payload, and additionally emits metadata_json, a serialized copy of that same payload with all the labels, scores, and frame assignments kept intact.

Why the second output? Because core bounding boxes are just coordinates - the label and confidence are gone. When you hand boxes to a core node and then need to know which region was "the cup," the metadata JSON is your lookup.

Inputs and outputs

• detections (VLM_DETECTIONS) - the only input.
• bounding_boxes (BOUNDING_BOXES) - what you wire into core nodes, region tasks, or crop/inpaint logic.
• metadata_json (STRING) - the same boxes plus labels/scores/frames, for downstream logic or debugging.

The pack's README calls out one clean use: Florence-2 region tasks (region-to-category, region-to-description, region-to-OCR) take a single core BOUNDING_BOX per image. Chain this node (or VLMSelectDetection) to get exactly one box into those tasks.

Installing this pack

Same as every node in this pack - one repo, install once. ComfyUI Manager: search VLM_nodes. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
cd ComfyUI_VLM_nodes
python -m pip install -r requirements.txt

Use ComfyUI's Python for pip; this node downloads no models.

Common issues

The thing to watch is coordinate space, and it's the same trap across the whole detection family: if your detections came from a downscaled or resized analysis copy, those boxes describe that copy. Feed the matching-resolution image to whatever consumes the boxes, or run them through the same coordinate frame - boxes in one space and an image in another is the classic "boxes are drawn in the wrong place" bug.