Qwen2.5-VL Object Detection

The whole pack exists for this node. Qwen2.5-VL Object Detection takes the model from the loader, an image, and a plain-English description of what you're hunting, and returns bounding boxes. Not "dog" and pray - "the small brown dog sitting on the left of the sofa" and get exactly that dog.

This is the modern replacement for the GroundingDINO step in the classic GroundingDINO + SAM pipeline. GroundingDINO is a decent zero-shot detector, but it falls apart on long or nuanced descriptions and gets famously confused when an image holds several similar characters. A real VLM understands natural language, and that's the whole difference between "works for 'cat'" and "works for 'the second cat from the right, the one with the collar'." Worth knowing that newer VLM+SAM hybrids like Sa2VA skip boxes entirely and emit masks directly - if that's what you want, use them. This pack is the "give me clean boxes, I'll handle segmentation myself" route, which plugs into the SAM2 workflows you probably already have.

Mechanically it's simple and robust. The node builds the prompt Locate the {target} and output bbox in JSON, runs it through Qwen's chat template, generates up to 1024 tokens, then parses the JSON back out of whatever the model actually said. That parser is forgiving on purpose - it strips ```json fences, digs through dict wrappers, and even recovers from truncated output. Boxes come back in normalized coordinates, get rescaled to absolute pixels against your image's real dimensions, are filtered by confidence, and are sorted best-first.

The inputs that actually matter:

• qwen_model and image - from the loader node and whatever you're probing.
• target - the entire job. A plain-English description of the object. It defaults to the uninspired "object", so type something.
• score_threshold - float 0–1, default 0. Drops boxes below a confidence score. Leave it at 0 and nudge up only if you're getting junk boxes.
• bbox_selection - all by default, or comma-separated indices like 0,2. Because boxes are sorted by confidence, 0 means "the most confident detection." Perfect when Qwen finds three cats and you only want the strongest one.
• merge_boxes - boolean that merges your selected boxes into a single bounding box. Great for "every person in this crowd" → one box.

Two outputs come out. text is the full JSON string, one {"bbox_2d": [x1, y1, x2, y2], "label": "cat"} entry per detection - handy for logging or anything that eats JSON. bboxes is the BBOX list of [x1, y1, x2, y2] absolute-pixel coordinates, which is what you actually feed forward. Route it through Prepare BBoxes for SAM2 and into kijai's segment-anything-2 nodes - or LayerStyle's SAM2UltraV2, which the pack's own example workflow uses - and a box becomes a proper mask for inpainting, compositing, or background work.

Two gotchas from people actually running this. First, VLM grounding boxes run tight - they wrap the object, not its halo. That's fine for SAM2, which works well from a tight box, but if you're converting the box straight into a mask for inpainting, pad it generously or the fill model will invent clutter around the edges. Second, the box indices are confidence-ranked, not spatial: "leftmost box" isn't index 0, "most confident box" is. And if the model's output won't parse as JSON at all, the fallbacks usually save you - a clearly hallucinated label means it needs a more specific target or a bigger checkpoint.

Installation is shared with the rest of the pack: ComfyUI Manager, search "ComfyUI Qwen2.5-VL Object Detection Node", install, restart. The real wait is the model download from the loader node - make sure that finished cleanly before you blame this one.