Gemini 图像编码器 (Inline)

The difference between a blind chat and a vision chat

Gemini can't read a torch tensor. GeminiImageEncoder is the adapter that turns an IMAGE from any sampler or loader into the inline base64 data that Gemini's API actually accepts. Without it, GeminiChat is talking about an image it can't see; with it, you can ask "what's wrong with this composition?" and get an answer grounded in what's actually on screen.

How it works

The node takes the first image of the input batch, converts it to a PIL image, encodes it to the format you pick, base64-encodes the bytes, and wraps them as {"inlineData": {"mimeType": ..., "data": ...}} - the standard shape Gemini's generateContent expects. Nothing goes to the network here; this is pure local encoding. The actual request happens later, when the content item reaches GeminiChat.

The fields that matter

• image - the IMAGE tensor. Only image[0] gets encoded.
• format - png, jpeg, webp, heic, heif. Default is jpeg, and that's usually right.
• quality - 10–100, default 85. Only applies to jpeg/webp.

The format/quality choice is a token-and-cost lever. JPEG at 85 is the sweet spot for photos and most renders: small request, fast, cheap. Switch to PNG when the image is text, line art, or has fine detail where compression artifacts would mislead the model. HEIC/HEIF exist mostly for completeness - they rarely matter unless a model demands them.

Outputs

content_item - the GEMINI_CONTENT_ITEM to wire into GeminiChat (directly or through a connector). base64_data - the raw base64 string, if you want to inspect or reuse it elsewhere.

The trap: batches

The node encodes only the first frame of the batch. Feed it a 16-frame batch and it silently sends frame 1 - no error, no warning. If you're iterating over frames in a loop or a batch, split the batch and run the encoder per image, or you'll spend a whole run believing Gemini saw the whole sequence when it saw one frame.

Other gotchas: RGBA images are flattened to RGB for jpeg automatically (good - jpeg has no alpha). Big images mean big base64 blobs and proportionally more input tokens, so downscaling before encoding is a legitimate cost hack. And no API key is needed here - this node is pure local work, so it works even before you've set up the loader.

Install

It ships in spawner1145/comfyui-aichat - ComfyUI Manager (search "comfyui-aichat") or:

cd ComfyUI/custom_nodes
git clone https://github.com/spawner1145/comfyui-aichat

Restart ComfyUI. Dependencies are just httpx, openai, and PyYAML; no model downloads. For one-off images, this inline path beats the File API - faster, nothing left on Google's storage.