Gemini Spatial Understanding

You feed this node an image, type what you're looking for ("the red cup", "a face"), and it tells you exactly where that thing is - either a 2D bounding box or a point - and hands back a copy of the image with everything drawn on it. That's the whole job, and it's the API-powered sibling of GroundingDINO: instead of installing and running a local detection model, a frontier vision model does the finding on someone else's servers.

One thing to get straight before you wire anything: the name says Gemini, but the call goes through OpenRouter, not Google. The node POSTs to https://openrouter.ai/api/v1/chat/completions with an OpenAI-compatible payload, so the key you need is an OpenRouter key, and the default model string google/gemini-2.5-flash is just that model as routed by OpenRouter. No Google Cloud project, no Vertex, no billing account - one key from openrouter.ai. If you already have an OpenRouter key sitting around from other ComfyUI API nodes, you're done.

How it works

The mechanism is straightforward and it's worth knowing, because it tells you what to expect from the output. The node takes the first image in your batch, converts it to JPEG, base64-encodes it, and builds a prompt like Task: Detect {target}. Return a JSON array... "box_2d": [ymin, xmin, ymax, xmax] (coordinates 0-1000)... Return ONLY the JSON. It sends that with the image to OpenRouter, asks for a json_object response, and then draws the results back onto your image - red rectangles with labels for boxes, blue dots for points.

The annotated_image output is that drawing, which makes it your instant sanity check: run it once, look at the boxes, and you'll see immediately whether Gemini found what you wanted and where it thinks it is. The json_output string is the raw JSON array, and that's the thing you actually pipe downstream.

The inputs that matter

There are eight, but a beginner sets four:

• target - what to find, plain English. The default "items" is vague on purpose; the results get dramatically better when you name the thing ("person", "dog", "left eye").
• task_type - 2D bounding boxes or Points. Boxes for anything maskable; points when you just need a location.
• model - default google/gemini-2.5-flash. This is a free-text string, and here's the sneaky part: because it's an OpenRouter call, you can put any vision model OpenRouter hosts in there. Swap in a Qwen-VL if you want a different flavor.
• api_key - paste your OpenRouter key here, or leave it blank and set the OPENROUTER_API_KEY environment variable instead (the node reads a .env file too).

The rest - temperature (0 by default, which is what you want for detection), reasoning_effort (none/minimal/low/medium/high/xhigh), and service_tier (default/flex/priority) - map straight onto OpenRouter's API. Leave them alone unless you know why you're touching them.

Install

The README is a single line and there are no model files to download - the "model" lives on OpenRouter's servers, and the only real dependencies are the ones ComfyUI already ships plus requests and python-dotenv.

cd ComfyUI/custom_nodes
git clone https://github.com/nova452/ComfyUI-Gemini-Spatial

Then restart ComfyUI. Or just search ComfyUI-Gemini-Spatial in ComfyUI Manager and hit install.

Common issues

• "OPENROUTER_API_KEY is required" - you didn't pass a key and there's no env var. The node raises this on purpose; it refuses to guess.
• HTTP 401 - the key is wrong, or it's a Google key. It needs to be an OpenRouter key.
• Coordinates look "off" - everything comes back in 0–1000 normalized space, not pixels. That's by design; the other nodes in this pack (and your own math) convert them.
• It only looks at the first image in a batch. Feeding it a video frame sequence analyzes frame zero and draws on that.

Two honest caveats. This is a metered API call - every run costs a fraction of a cent and sends your image to OpenRouter's servers, so it's for "I need to find a region" jobs, not for scanning a thousand frames. And the pack is a 0-impression, single-commit personal project with essentially no documentation - the pattern it implements (VLM boxes feeding an inpaint/detail loop) is well-proven, but you're relying on a small wrapper around a big API.