Crop VLM Detections

Open-vocabulary detection is the gift that keeps giving and also the gift that gives you forty boxes full of pixels. Sometimes you want the boxes drawn on the image; other times you want the pixels out of the image - a crop per detection, cleanly separated, so you can do something with each object individually. Crop VLM Detections is the second of those.

It's a detection-utility node in gokayfem's VLM_nodes pack, sitting right after anything that emits VLM_DETECTIONS - the open-vocabulary detector, the structured spatial parser, Moondream 3's detect node, Florence-2. Any of those feed this node, and it produces a batch of cropped images plus metadata describing what each crop is.

How it works

You give it the source image and the detections. For each detection it cuts out the bounding box from the matching frame, applies the padding you asked for, and (optionally) pads the result to a square. Output is an IMAGE batch of crops plus a crop_metadata_json string that maps each crop back to its detection - label, score, source coordinates, and frame.

That metadata matters more than it looks: when crops come out in a batch, you need to know which crop is "the person" versus "the car." The JSON is your index.

Inputs and outputs that matter

• image (IMAGE) - the frame(s) the detections refer to. Detections and image must be in the same coordinate space, which they are if both came from the same pipeline.
• detections (VLM_DETECTIONS) - the boxes/polygons to crop.
• padding - margin added around each box, in pixels. Default 0; a few pixels of context usually makes downstream analysis much better.
• square - pad crops to a square instead of leaving them box-shaped. Handy when you're feeding them to a VLM that expects a fixed aspect.

Outputs: crops (IMAGE) and crop_metadata_json (STRING).

What to do with the crops

The classic move is feeding each crop to a second VLM pass - region captioning, OCR on just the text regions, identifying a specific detected object. It also feeds the pack's VLMTrackAwareCrops philosophy: label identities from representative crops instead of rereading every full frame. And for plain human inspection, a crop grid is far easier to scan than a wall of boxes.

Installing this pack

It's one pack, so the install is the same as the other nodes here. ComfyUI Manager, search VLM_nodes, install. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
cd ComfyUI_VLM_nodes
python -m pip install -r requirements.txt

ComfyUI's Python for pip, no model downloads for this node.

Common issues

Coordinate mismatch is the one that bites: if the detections came from a downscaled analysis copy and you crop the full-resolution image, every box is off by the scale factor. The pack's pixel budget node changes resolution deliberately, so if you use it, remember the detection coordinates are in its output space - crop that same resized image. Padding larger than the box can push past the frame edge on objects at the border; the node clips, but tight objects near the edge are where you'll see unexpected behavior.