🔍 Gemini Object Detection

Gemini Object Detection is the "find things in a picture" node that doesn't need a downloaded model at all. You type what you're looking for in plain English - "detect all people and vehicles" - and Google's Gemini looks at the image over the API and hands back bounding boxes, labels, and masks. No YOLO weights, no Ultralytics install, no AGPL licensing question hanging over your workflow. The trade is that it's a cloud call: it needs a Google API key, it costs per image, and your image leaves your machine.

That trade is worth making in two situations. First, when your detection classes are arbitrary - Gemini understands natural language, so "find the color-matching chart and the hang tag" works without training a detector on them. Second, when you're already in the closed-model world and don't want to bolt a whole local detection stack onto the side of it. If you want the local, free, privacy-preserving alternative, the KB's detection doc lays out the YOLO/MediaPipe landscape; this node is the no-install cloud lane.

How it works

The node sends your image plus the detection_prompt to Gemini, gets back a set of detections with bounding boxes and labels, and does the drawing locally with PIL - boxes, labels, and optionally the annotated image only, JSON only, or both (output_mode). It also converts each detection into a mask and unions them into a single union_mask, which is what makes this slot into the detect → crop → refine → stitch loop from the KB: wire masks into Gemini Mask Crop, refine the crops with Gemini Batch Refine, and paste back with Gemini Mask Stitch.

Two things worth knowing before you queue. First, the seed isn't just for reproducibility - it's a cache key. The node hashes image bytes + seed + params (IS_CHANGED), so the same image with the same seed skips the API call entirely and serves the cached result. That's a real money saver when you're iterating on annotation style. Second, model choice is a real speed/accuracy dial: the default gemini-3.1-pro-preview has the strongest spatial accuracy, 3.5 Flash is faster, and the 2.5 models are the stable fallback if the newer ones act up.

The inputs that matter

• detection_prompt - the whole point. Be specific: "Detect all people and vehicles" beats "Detect all objects."
• model - default 3.1 Pro. Drop to a Flash tier for bulk runs.
• output_mode - annotated_image, json_only, or both. For piping into a workflow you want both; for data extraction, json_only saves the drawing step.
• annotation_style - boxes, boxes_labels, or labels_only.
• confidence_threshold - default 0 keeps everything; raise it to filter weak detections. Note it filters, but Gemini doesn't give you calibrated confidences the way YOLO does, so treat it loosely.

Outputs: annotated_image, masks, union_mask, detections_json, and a summary string.

Install, key, and gotchas

Same pack as the rest of the suite: ComfyUI Manager search "ComfyUI_Gemini", or clone into custom_nodes and pip install -r requirements.txt - google-genai (not google-generativeai), python-dotenv, scipy. Put GOOGLE_API_KEY in the pack's .env so the key stays out of your workflow files. Common complaints: "API key is missing" (check the .env placeholder), safety blocks on sketchy prompts, and preview-model names getting retired. And budget - this pack is a small single-author project, so when something breaks, the console logs are your support ticket; they're detailed, which helps.