LocateAnything Grounding

Type person into a box, hit run, and every person in the image comes back with a rectangle around them. That's the whole pitch of LocateAnything Grounding - it runs NVIDIA's LocateAnything-3B visual grounding model and is the closest ComfyUI has to "ask what's in the picture, get coordinates back."

The killer use is text-grounded masking. The modern inpainting pattern is: name the object, get its bounding box, pad it a bit, and let Flux Fill or whatever you're using redraw just that region. The crop-and-stitch crowd lives and dies on that flow, and until recently "get the box" meant a clunky SAM + GroundingDINO setup or an API call. This node does it in one step, on your GPU, with no key. It also does GUI grounding, scene-text localization, and document layout analysis.

How it works

Under the hood it's a vision-language model - a Qwen-based checkpoint. The node builds a prompt from your task + query (e.g. ground_multi becomes "Locate all the instances that match the following description: …"), feeds the image through a processor, and gets back a text answer with coordinates in the model's normalized [0, 1000] format, which it scales to pixel coordinates and draws. A video loaded as an IMAGE batch is processed frame by frame with a native progress bar - frame N gets seed + N if you've turned temperature up.

Two behaviors worth knowing. First, this is a grounding model, not a segmentation model: boxes become rectangular masks, points become circular masks. You get a filled-in box, not a silhouette. Need contours? Chain a SAM node downstream and feed it the mask. Second, it's defensive about failure - garbage output or zero coordinates triggers one automatic retry in slow mode. And a gotcha baked into the code: on CUDA, temperature > 0 isn't really a thing - the node forces safe sampling (temperature=0.0, top_p=1.0, top_k=0) because higher temperatures trip device-side asserts. Leave it at 0 and don't fight it.

The inputs that actually matter

• task - the mode. ground_multi (every match), ground_single (one), detect (object categories), ground_text / detect_text (text in the image), gui_box / gui_point (GUI automation), point, and custom (sends query as the whole prompt; VQA is experimental here).
• query - what you're looking for: person, red car, the logo. Ignored in detect_text mode.
• generation_mode - hybrid (recommended default), fast (MTP parallel decoding, quickest), slow (autoregressive, most robust). Stubborn prompt returning nothing? Drop to slow.
• max_new_tokens - bump toward 8192 for dense scenes; the model card recommends it to avoid truncation.
• mask_grow / mask_blur - expand or shrink the mask and soften edges. The inpainting pattern calls for growing it - grounding boxes hug the object tighter than you expect.
• overlay_color / overlay_opacity - the preview blend color (hex like #00ff66) and its opacity.

It even has a ? icon in the title bar that opens the full parameter reference in-app - a nice touch.

Outputs

Five. annotated_image (boxes/points drawn on the frames) for eyeballing results, mask (the grow/blurred box-and-circle mask batch) into any mask input - crop, inpaint, mask preview - mask_overlay for a colored preview, plus answer and locations_json (prompt, timing, normalized and pixel coordinates, batch_index) for scripting. locations_json is a gift for automation: structured, not a regex hunt.

Install

Via ComfyUI Manager (search ComfyUI-LocateAnything) or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/alisson-anjos/ComfyUI-LocateAnything.git
cd ComfyUI-LocateAnything
pip install -r requirements.txt

Restart ComfyUI. Then the first run of the Model Loader downloads ~7.8 GB into ComfyUI/models/LocateAnything/.

Issues and gotchas

• License: the checkpoint is NVIDIA's non-commercial license - academic/research only. The integration code is GPL-3.0. Read the model card before building anything commercial on it.
• No coordinates at all? Try generation_mode: slow with temperature: 0; if it persists, reload the model with attention: eager. The node's own log tells you to do exactly this.
• trust_remote_code: the checkpoint ships Python that ComfyUI executes at load time. The node also patches known API mismatches with newer transformers versions automatically - which is why it keeps working when other remote-code nodes rot.
• CPU works but crawls. This is a 3B VL model; NVIDIA is the intended path.
• VQA in custom mode can return plain text and an empty mask - expected, since the model is optimized for localization, not chatting.

The pack is young - NVIDIA shipped LocateAnything-3B in spring 2026 - so the community is still mapping its failure modes. For turning words into boxes and boxes into inpaint masks, it's the one I'd reach for.