Academia SD Gemini Vision 👁️✨

Sometimes you don't want to run a local vision model at all. The cloud ones are just better at describing a scene, and if you're building a dataset and have a Gemini API key burning a hole in your pocket, this node lets you send images straight to Google's vision models from the canvas - no local weights, no VRAM cost, just an HTTP call and a caption back.

How it works

You feed it an instruction (the default is a JSON-output prompt that asks for structured data, including bounding boxes), optionally an image, and it calls the Gemini API with your api_key. The model dropdown lists current Gemini and Gemma variants, and the node can fetch your account's available models dynamically if you want something not on the list. The response comes back on the text (STRING) output, ready to route into a caption saver or any downstream text consumer.

Two things make it feel designed rather than bolted on: the token is saved to models/academia_tokens.json so you don't paste it every run, and the optional width/height inputs let the node rescale any bounding-box output from the model's normalized 0–1000 coordinates into pixel coordinates for your actual image - a real time-saver if you're using Gemini for detection-style dataset work rather than plain captioning. external_prompt lets you inject context from another node.

Install

Pack install: ComfyUI Manager → "AcademiaSD", or git clone https://github.com/AcademiaSD/comfyui_AcademiaSD, restart. Then the manual step - this node needs Google's SDK, and the pack has no requirements.txt, so it won't install itself:

pip install google-genai

The node checks for the library on load and tells you if it's missing. You'll also need a real Gemini API key from Google AI Studio. The default model in the dropdown (gemini-3.5-flash) is a current cheap-and-fast option; the flash-lite tier is usually plenty for captioning.

The honest trade-off

The appeal is obvious: better captions than a 2B local Qwen model, zero VRAM, no multi-gigabyte downloads. The cost is a cloud dependency and per-image API spend, plus your images leaving your machine - a real consideration if the dataset is sensitive. For bulk captioning at scale, cloud vision gets expensive fast, and that's when the pack's local AcademiaSD VLModel (Down)Loader + LLM Vision pair becomes the right call. For a quick, high-quality captioning run on a few hundred images, this is the one you'll reach for.