Apply Gemma3

ComfyUI is great at generating images and famously bad at telling you what's in them. ApplyGemma3 works the other direction: it hands an image - plus a prompt - to Google's Gemma 3 vision-language model and returns plain text. It's a captioning/interrogation node in the truest sense. You feed it a model and processor from the pack's loader, plug in any image, and out comes a description string you can route anywhere a string goes in ComfyUI.

The use cases that actually justify it: captioning a dataset before LoRA training, getting a second pair of eyes on why a render came out wrong, auto-writing a prompt from a reference image, or just describing an image into a text field without alt-tabbing out of ComfyUI.

How it works

Under the hood it's a thin chat-inference wrapper. It builds a message list with a hard-coded system prompt ("You are a helpful assistant."), appends your prompt - and the image, if you connected one - as the user turn, runs the model's chat template through the processor, and decodes the generated tokens into a string. Nothing exotic.

Two things worth knowing before you use it. First, generation is greedy: the code calls generate with do_sample=False, so there's no temperature, no sampler, no seed. Same prompt + same image = same answer, every single run. That's great for reproducible captioning and useless if you wanted variety. Second, it's not a batch node - the image input is a single IMAGE (the code squeezes the batch dimension), so feed one image at a time.

The inputs that matter

Only a handful, and they're all self-explanatory:

• model, processor - the MODEL and PROCESSOR from Gemma3ModelLoader. These are pack-local types; nothing else in ComfyUI produces them.
• prompt (STRING, default "Describe this image in detail.") - your instruction. Gemma is instruction-tuned, so prompt it like a boss.
• max_new_tokens (INT, 1–1000, default 100) - the cap on generated tokens. For a rich caption, 100 is stingy; the ceiling is 1000, so give it room.
• image (IMAGE, optional) - leave it disconnected and you get a plain text-only chat. Connect it and the model actually looks.

Wiring it in

The single output is description (STRING). The most common sink is a CLIP Text Encode node's text input - the model's description becomes a diffusion prompt - or any text-saving/display node. The node itself isn't an output node, so you'll want to display or save the string somewhere or it just vanishes when the queue finishes.

Install

Same story as its sibling node: search "Gemma3" in ComfyUI Manager, or

cd ComfyUI/custom_nodes
git clone https://github.com/leeguandong/ComfyUI_Gemma3.git
pip install -r ComfyUI_Gemma3/requirements.txt

then restart ComfyUI. Read the loader article first, because the requirements file pins a dev branch of transformers - that's the pack's real pain point, and it'll bite you here too.

Where it bites

• First run crawls. This node downloads nothing itself - all the weight sits in the loader, which pulls 5–50+ GB and loads it into memory. Be patient on run one.
• VRAM. While this node runs, the whole LLM stays loaded. On an 8–12 GB card stick to the 1B or 4B Gemma, and expect it to fight the diffusion model for memory.
• Truncation. At the default 100 tokens, a long description gets chopped mid-sentence. Bump max_new_tokens.
• No knobs. Deterministic, greedy, fixed system prompt. If you want temperature and sampler controls, look at a fuller VLM pack (Joy Caption, an Ollama node); this is a quick, honest wrapper, not a control room.