Gemini Pro

The pitch: one node that pipes your ComfyUI graph into Google's Gemini API and gets a real language model to actually look at what you're making. No local model download, no VRAM, no checkpoint hunt. Drop in an image or two, type a question, and get a STRING back that you can route anywhere - into a captioner, a prompt-builder, a text-concat, wherever you like.

It's the flagship of a three-node pack from penposs, a Bilibili-based developer. Honest framing up front: this is a small, hobbyist pack, not a community staple - its README is bilingual Chinese/English and its footprint is tiny. And the README oversells a bit: it talks about image generation, but the shipped code does multimodal understanding. Text out. That's the whole deal.

What it is and when you'd reach for it

ComfyUI is famous for local diffusion, but half of a serious workflow is understanding what you made - captioning a batch, describing a video, asking "what actually changed between these two frames?" That's what this node does. It's an API node: it calls Gemini's API with whatever you hand it and returns text. You'd pull it in when you want a vision-language model mid-graph - auto-captioning a dataset, or a QA step that describes a render before you decide to keep it.

The Gemini free tier works for light use, though it rate-limits hard - which is exactly why the code looks the way it does.

How it works

Under the hood it's a thin wrapper around google-generativeai. It configures the SDK with your api_key, builds a content list from your prompt plus attached media, and calls generate_content. The seed does something cute: instead of randomizing sampling, it appends a hidden tag like [variation-12345] to your prompt so a different seed genuinely produces different output. It also persists your api_key to a config.json next to the node - first run saves it, later runs just load it.

Then there's the retry machinery. The code sleeps before every call, retries with exponential backoff on quota errors, and enforces a 30-second timeout. This node is basically a monument to the HTTP 429. None of it is fast - budget a few seconds per call even on a good day.

The inputs that matter

Most of the required inputs are self-explanatory; here are the ones you'll actually set:

• prompt and system_prompt - what you're asking, and the persona/prepended instructions.
• input_type - text, image, video, or audio. Pick the mode for what you're feeding in.
• model - gemini-2.5-flash-preview-09-2025 by default, with gemini-3-pro-preview and an older 2.5-flash preview on offer. 3 Pro is the fancy one (the engine behind the Nano Banana image models); 2.5 Flash is cheaper and snappier for plain analysis.
• api_key - grab one free at aistudio.google.com. If it's blank and no config.json exists, the node returns an error string instead of crashing.

The optional inputs that matter: image1 through image4 for multi-image input (the node converts ComfyUI image tensors to PIL and hands up to four to the API), plus video (expects a file path string, not a ComfyUI VIDEO object) and audio. Rounding it out: temperature, max_output_tokens, delay_time (a deliberate sleep before the call), proxy, and seed.

Output: exactly one STRING. Wire it into a Show Text node and read what Gemini said.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/penposs/ComfyUI_Gemini_Pro.git

Or search "ComfyUI Gemini Pro" in ComfyUI Manager and let it handle it. Restart ComfyUI either way. Installing the pack gives you all three nodes at once.

The pack's requirements.txt pulls google-generativeai, google-genai, sounddevice, and openai - the last two are imported nowhere and are pure cargo. A taste of the pack's polish, and a hint at why the README tells you to uninstall and reinstall google-generativeai if you hit conflicts: this SDK fights with packages that pin older versions of it.

Common issues

• No key, no output. If api_key is empty and config.json doesn't exist, you get a Chinese error string back in your text output, not a crash. Set the key on the node once and it sticks.
• Rate limits. 429 / "Resource has been exhausted" is this pack's natural habitat. Free tier quotas are per-day and tight; the retries help, but the real fix is a paid key or patience.
• Safety blocks. The code checks prompt_feedback.block_reason and reports when Gemini's filters swallow your request. It's not you breaking anything.
• Proxy. The author is behind the Great Firewall and the README is bilingual for a reason - if you need one, proxy works, and the node auto-prepends http:// when you forget.

One last note, because it matters in this ecosystem: custom nodes run arbitrary Python on your machine, and there's been at least one prosecuted malware case via a node pack (ComfyUI_LLMVISION). This one is small, MIT-licensed, and only talks to Google - but the habit of reading what you install is worth keeping.