Detect By Label

Detect By Label is the "find the thing, mask the thing" node. You feed it an image, it runs an Ultralytics YOLO (or YOLO-World) model over it, and it hands you back a mask covering everything it found - plus the label names, the bounding boxes, and an annotated preview. If you've ever wished the ADetailer detect-and-inpaint loop had a ComfyUI equivalent, this is the detection half of it. It's from the same author (shadowcz007) as the popular comfyui-mixlab-nodes, and it lives in the Mixlab/Mask category for a reason: the mask is meant to drive region-based inpainting downstream, the "fix the face, leave the rest alone" workflow that's been a staple since the ADetailer days.

How it works

Under the hood it's a thin wrapper around the ultralytics Python package. You pick a .pt weight file and a mode - YOLO-World or YOLOv8 - and on every run it loads the model fresh, runs detection on your image, then converts each detected box into a mask. With a YOLOv8-seg model you get proper object-shaped segmentation masks; with a plain YOLOv8 detector it falls back to drawing a rectangle for each box. All masks are then merged into a single mask per image, and the node returns the merged result, the class names, and the box coordinates as grids.

The open-vocabulary part is the genuinely fun bit. A regular YOLOv8 only knows its training classes (the usual COCO 80). YOLO-World accepts arbitrary text at inference time, so with yolov8s-worldv2.pt you can type "cat" or "red car" or "wristwatch" into target_label and it'll find them without retraining anything. That's why this node beats a fixed detector for anything that isn't a face.

The inputs that actually matter

• model - the dropdown lists every .pt file found in ComfyUI/models/ultralytics at startup. Grab weights from the ultralytics assets releases: yolov8s.pt for plain detection, yolov8s-seg.pt for object-shaped masks, or a yolov8*-world*.pt for open-vocabulary.
• type - YOLO-World or YOLOv8. It must match the weight file; running a world model as plain YOLOv8 won't do what you think.
• confidence - default is 0.1, which is very low. You'll get boxes on almost anything, so expect noise and lean on target_label to filter. Crank it to 0.3–0.5 for cleaner results.
• target_label - optional. A comma/space-separated list of class names to keep; leave it blank to keep everything. The match is exact and case-sensitive, so person won't catch Person.

Outputs: masks (one merged MASK per input image), labels (STRING list), grids (Mixlab's _GRID boxes, x,y,w,h), and image - the annotated debug overlay, which only fills when debug is set to on.

Installing it

ComfyUI Manager handles it - search comfyui-ultralytics-yolo. Or clone it yourself:

cd ComfyUI/custom_nodes
git clone https://github.com/shadowcz007/comfyui-ultralytics-yolo

Then restart ComfyUI. The only Python dependency is ultralytics (installed by the bundled install.bat, which uses the Tsinghua PyPI mirror - if that's slow for you, just pip install ultralytics manually). The step everyone forgets: drop your .pt files into ComfyUI/models/ultralytics and restart again - the model dropdown is built once at startup, so new files won't appear until you reload.

Where people get burned

• Empty model dropdown - you installed the pack but no weights. There's no bundled model; nothing works until you add a .pt.
• Merged masks. All detected regions collapse into one mask per image. If you need a separate mask per object, this isn't the node - it's a single-region inpainting feeder, not an instance segmenter.
• "Nothing found" still outputs a mask - a black (all-zero) mask with label - and a full-image grid. If you're not careful, an empty detection silently feeds an all-black mask into your inpaint and you wonder why nothing changed.
• Slow repeated runs - the model is reloaded and discarded on every execution (del model at the end). Fine for a one-shot fix, annoying in a live workflow.

One honest caveat: this pack is tiny and last touched in mid-2024, so treat it as a stable-but-dormant utility rather than an actively evolving one. For the mask you need right now to patch a face or swap a tiled background, it's still one of the quickest roads from "image" to "inpaintable region" in ComfyUI.