Expanded Gemini Interactions (Text/Image)

This is the node that actually does something: it sends your prompt - plus any images and video you wired in - to Google's Gemini Interactions API and drops the answer back as a string. The API-wrapper pattern in its purest form, a node that looks like a generator and is really an HTTP client with your key attached (external-api-nodes.md).

Three jobs. Rewrite a rough idea into a structured prompt before your local sampler - the honest ceiling is "removes the blank page", not "writes better than you" (llm-in-comfyui.md). Read an image and describe it, for img2img seeding or caption drafts. Or generate an image outright: point model at an image model, flip include_images, and you're calling the Nano Banana line - technically excellent, aggressively filtered (closed-source-models.md, kg/entities/google.json).

On the author: silveroxides is a quantisation-and-LoRA name here who expanded tatoookan's original Gemini repo. The pack has no Reddit footprint, so everything below comes from the shipped code.

How it works

It calls client.interactions.create(...) - the newer Interactions surface, not the classic generate_content - with inputs assembled as ordered parts: video first, then images, then your prompt. A response format list rides along, built from include_images + aspect_ratio + image_size when you've picked an image model, and from a connected response schema when you've defined one.

Before any of that it fingerprints everything - prompt, system instruction, model, token cap, hashed image tensors and video bytes, safety settings, schema JSON, seed, even proxy host and port - and checks a local result cache. Two consequences: the cache is only consulted when use_seed is on, and with a fixed seed and an unchanged prompt, re-queueing returns the previous answer from memory without touching Google. Bump the seed for a new call.

The request runs in a background thread with timeout (15–300s) around it. When it expires the node puts either timeout_fallback_text or a standard "Gemini API request/processing timed out" line into text and moves on - no exception, no red node, just text that isn't an answer.

Inputs that matter

config is the credential socket from Configure Gemini Interactions API Key; nothing works without it. prompt is your ask. system_instruction defaults to "You are a helpful AI assistant." and is where you actually steer behaviour for prompt-rewriting work - the field people forget exists. model defaults to gemini-2.5-flash, and the long list spans text, thinking and image models. max_output_tokens goes to 65536.

Three widgets are traps, and the author says so in the tooltips: temperature, top_p, top_k and thinking_budget are all marked "Ignored in Interactions API". Moving them changes nothing except the cache fingerprint. Randomness control lives in seed; reasoning effort lives in thinking_level (None through high - some models reject minimal and the node swaps in low with a warning).

For vision work: image_inputs is an autogrow socket set growing from image_1 to image_100, batches sent consecutively, and media_resolution sets input resolution for images, video and PDFs. video takes a GEMINI_VIDEO_CONFIG from the pack's video node. response_schema - labelled Response format - requests JSON, returned in both text and Structured output.

Multi-turn is two widgets: store (default true) keeps the interaction server-side, previous_interaction_id continues from a prior one. max_retries with retry_pattern is the re-roll mechanism - if the response text or an error matches your regex, the node retries with a new seed. Every retry is a billable call.

bypass_mode (system_instruction, prompt, or both) rewrites your text with homoglyphs and zero-width joiners - periods, spaces, commas, quotes swapped for lookalikes - so naive keyword filters don't match. It changes what the model reads, can break exact formatting, and won't get past the model's own refusals. Last resort.

Outputs

text is the answer - and the error channel. Failures, timeouts and blocks all come back as strings here, so a downstream node will happily consume your error message as content.

image is a real generated image when image generation was enabled; otherwise it's a dark grey 64×64 placeholder, still perfectly connectable - which is how people wire a fake image in and wonder why the output is grey. final_actual_seed is the seed the API actually used. Structured output holds parsed JSON when a response format succeeded. Thoughts carries returned thinking when include_thoughts is on. Interaction ID feeds back into previous_interaction_id for a follow-up turn.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI_Gemini_Expanded_API.git
cd ComfyUI_Gemini_Expanded_API
pip install -r requirements.txt

Then restart. The requirements floor is google-genai>=2.17.0; an older copy left behind by another pack shows up as 'Client' object has no attribute 'interactions' in text. import cv2 is used but not listed - stock ComfyUI ships OpenCV, a bare Python env doesn't.

Where people get burned

• Re-queuing gives the identical answer. Cached, by design, when use_seed is on and nothing changed. Randomize the seed.
• Errors arrive as text, not as failures. Timeouts, bad keys, policy blocks - all string output, and your downstream prompt-formatter will forward them.
• include_images plus a non-image model does nothing. The response-format branch only fires for models in the image list, and image generation plus a connected response format is an error rather than a silent choice.
• image_size is model-specific. The node validates it and names the allowed values.
• Policy refusals look like bugs. The README's NoneType object has no attribute 'parts' note is a prohibited-use refusal; the Interactions path reports blocks as Interaction blocked/failed instead. Test with ordinary subject matter first.
• The README is a step behind. It documents the older non-Interactions text node (Configure Gemini API Key, different api_version list, no Interaction ID output). Here the tooltips are the documentation.