ComfyUI Node

HT Gemini

A chat model inside your graph, with image understanding

By ArtHommage·Created 2 years ago·Updated about a year ago· 4
HT Gemini
  • image
  • result
  • status
modelgemini-2.0-flash
refresh_modelsfalse
system_instructionsYou are a helpful assistant.
content
temperature0.70
top_p0.95
top_k40
max_tokens2048
response_mime_typetext/markdown
safety_settings{"HARM_CATEGORY_HATE_SPEECH": "BLOCK_NONE", "HARM_CATEGORY_SEXUALLY_EXPLICIT": "BLOCK_NONE", "HARM_CATEGORY_DANGEROUS_CONTENT": "BLOCK_NONE", "HARM_CATEGORY_HARASSMENT": "BLOCK_NONE"}

Most of ComfyUI is about making images, but a surprising amount of it is about reading them - captions, prompts, structured analysis of what's in a frame. HT Gemini is the pack's hook into Google's Gemini text models for exactly that: it takes a text content prompt, optionally an image, and returns the model's answer as a STRING you can wire into anything else. Want a node that looks at your render and writes the prompt for the next pass? This is the piece.

How it works

It's a thin wrapper around the google-generativeai SDK. You get a model dropdown with eleven options, from gemini-2.0-flash (the default, and the sensible workhorse) through gemini-2.5-pro-preview and gemini-2.0-flash-thinking-exp, plus the 1.5 line and the 2.0-flash-lite budget option. refresh_models re-pulls the live list from Google instead of using the cached default. Like its image sibling, it needs a GOOGLE_API_KEY - environment variable, Colab userdata, or a GOOGLE.key file - because it's a real cloud API call, not a local model.

The inputs that matter:

  • system_instructions - the system prompt. Default is "You are a helpful assistant." This is where you actually steer it: "You are a strict captioner. Output only tags." That pattern - instruct, then pass content - is the whole trick of using this node well.
  • content - your question or the text you want processed.
  • image (optional) - drop an IMAGE in and Gemini becomes multimodal: describe it, count objects, read text off it, turn it into a prompt.
  • temperature, top_p, top_k, max_tokens - the standard sampling dials. For captioning you'll often want temperature low and max_tokens generous.
  • response_mime_type - text/plain or text/markdown (the default). Markdown is nice when you're asking for structured output that lands in a text display node.
  • safety_settings - and here's a spicy default: it ships with all four harm categories set to BLOCK_NONE. That's deliberate (you don't want Google's filter silently refusing to describe a generated image) but it means the onus for content policy is entirely on you.

Outputs are result (the model's text) and status (a status/error string).

What to do with it

The natural workflows: image → Gemini → caption → next prompt (automated positive/negative prompt generation for a batch), image → Gemini → structured JSON → parameter extractor → control nodes (let it name the subject, the lighting, the camera), and prompt evaluation loops where a second generation critiques the first. The STRING output plugs straight into any text-input node in the graph.

Installing and caveats

Standard pack install:

cd ComfyUI/custom_nodes
git clone https://github.com/ArtHommage/HommageTools.git
cd HommageTools && pip install -r requirements.txt

restart, and make sure GOOGLE_API_KEY is set in the environment that starts ComfyUI. The requirements pull in google-generativeai, so no separate step beyond that.

The honest version: this is a paid, internet-dependent, closed service bolted onto a local tool - the community's whole reason for going local is partly "no prompts leaving the machine," and this inverts that for whatever passes through it. It's fantastic for one-off analysis and prototyping. For production, think hard about cost, the filter-on-by-default-you-set-it-to-off model, and the pack's "perpetually in alpha" disclaimer before you make it a permanent workflow dependency.

CategoryHommageTools/AI

Inputs (11)

NameTypeDefaultDescription
modelCOMBOgemini-2.0-flash11 options: gemini-2.0-flash, gemini-2.0-flash-lite, gemini-2.5-pro-preview, gemini-2.5-flash-preview, gemini-1.5-pro, gemini-1.5-flash, +5
refresh_modelsBOOLEANfalse
system_instructionsSTRINGYou are a helpful assistant.
contentSTRING
temperatureFLOAT0.700–1
imageoptIMAGE
top_poptFLOAT0.950–1
top_koptINT401–100
max_tokensoptINT20481–8192
response_mime_typeoptCOMBOtext/markdown2 options: text/plain, text/markdown
safety_settingsoptSTRING{"HARM_CATEGORY_HATE_SPEECH": "BLOCK_NONE", "HARM_CATEGORY_SEXUALLY_EXPLICIT": "BLOCK_NONE", "HARM_CATEGORY_DANGEROUS_CONTENT": "BLOCK_NONE", "HARM_CATEGORY_HARASSMENT": "BLOCK_NONE"}

Outputs (2)

NameTypeDescription
resultSTRING
statusSTRING