Generate BBOXes

Every segmentation pipeline needs a detector, and usually the detector is the annoying part to set up. Generate BBOXes skips the detector entirely: it shows your image to a vision LLM, asks it where "dog" is, and gets back coordinates in the model's own grounding ability. The output is a proper bounding box you can hand straight to a SAM-style segmentation node, plus a preview image with the boxes drawn on in red so you can see what the model thinks it found.

How it works

The node sends the image with a fixed grounding prompt: locate the item, reply in <bbox>x_min y_min x_max y_max</bbox> format, coordinates as percentages 0–1000, list every instance. It then regex-scrapes the <bbox> tags out of the model's reply, converts the 0–1000 per-mille coordinates into real pixels, and converts them to x/y/width/height. Finally it draws red rectangles on a copy of the image. Everything you see in the preview is also in the data.

Inputs that matter:

• client_info - from the API LLM Loader. This one genuinely needs a vision model with decent grounding skills; a weak VLM will still answer, just with worse boxes.
• items - the thing to find. The default is dog. Keep it to a single clear noun phrase; the prompt handles multiple instances of one item, so "person" will box every person.
• image - what to search.
• unload_model_after_chat - Ollama only, unloads the model from memory after the call (keep_alive: "0").

Outputs:

• bbox (BBOX) - the coordinates as x, y, width, height. This is what you wire into other tools.
• BBoxPreviewImage (IMAGE) - the annotated image, useful as a visual check or as a condition image.

What it wires into

The README's whole point is compatibility: the bbox output is designed to plug into the bbox inputs on ComfyUI_LayerStyle nodes, and from there into Segment Anything. That's the classic grounded-editing chain - detect the subject with the VLM, segment it precisely with SAM, then inpaint, outpaint, upscale, or swap that region without touching the background. If you've used Impact Pack's BBOX/SAM detector providers, this is the same idea with an LLM standing in for the object detector. The coordinates come out in pixel-space x/y/w/h, which is exactly the format the segmentation nodes expect, so the handoff is clean.

Install and gotchas

Same pack install as everything else - ComfyUI Manager search ComfyUI-PDiD-LLM-Nodes, or clone it into custom_nodes, restart, and run ComfyUI 0.3.51+ (this pack is built on the new V3 schema, so older builds won't load it). Nothing to download; the model is behind your API client or local Ollama.

Where people get burned:

• Non-grounded models produce junk boxes. If your LLM can't follow spatial instructions, you'll get boxes around nothing, and the node will happily draw them. Test on one image before you trust it in a batch.
• Format-breaking replies. The parser requires the exact <bbox>…</bbox> format. If the model goes off-script you get the source's literal error - BBox is invaild. Please retry again. - and it's worth just re-running, because it's non-deterministic. This is an LLM doing object detection; some percentage of tries will be sloppy.
• It's an LLM, not a detector. Grounding quality varies wildly by model and by how visually distinct the object is. For hard detection workloads you'd still reach for a dedicated detector; this node is for when you want "the dog, approximately" wired into a segmentation graph without installing one.