Gemma 4 Multimodal

This is the node the pack is named for: a single Gemma 4 12B that accepts text, an image, video frames, or any combination, and answers in natural language - all local, no API. Where its siblings are single-purpose, this one is the Swiss-army version. Wire in a text prompt for a rewrite, plug an image for a caption, drop a frame batch for a description, or do two at once. Same model, same sampling settings, one node to learn instead of three.

The practical reason you'd reach for it over the dedicated nodes: workflow flexibility. One saved node does whatever the current graph needs, and its inputs are all optional, so you can build a single reusable "ask Gemma" block and feed it differently per job. The honest downside is that it's not the best choice for any one task - if you're always captioning, the image node's defaults fit better.

How it works

Identical core to the rest of the pack: it loads gemma-4-12b-it from ComfyUI/models/LLM/gemma-4-12b-it/ through the official ModelScope transformers implementation (AutoModelForMultimodalLM + AutoProcessor), sidestepping the GGUF/llama.cpp incompatibility that ate other Gemma 4 loaders at launch. It builds a chat from your optional prompt (system) and text (user), and prepends whatever media you connected - and here's a detail from the source worth keeping: images are placed before the text in the message, which the code comments flag as "Gemma 4 best practice." Video frames follow the image. If you feed both an image and a frame batch, it sees image then video then question.

The inputs that matter

• text - your request, the user message. Always relevant, even with media attached.
• prompt - optional system prompt that steers tone and format.
• image / video_frames - optional IMAGE inputs. Hook up one, both, or neither; text-only works fine.
• vision_token_budget - how many tokens the vision encoder spends on media. 280 default; raise for detail-heavy asks, lower for speed.
• enable_thinking - Gemma's reasoning pass, returned wrapped in <think>...</think>. Fine for complex questions; overkill for a caption.
• keep_model_loaded - default true, model stays resident between runs. This is the one to flip off when a 24GB model and your sampler both want the card.

Output is a single RESPONSE string. From there it feeds a prompt builder, a text save, or whatever string slot your workflow has.

Installing it

ComfyUI Manager, search "Gemma 4 - Multimodal AI", or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/mailzwj/ComfyUI-Gemma4
cd ComfyUI-Gemma4
pip install -r requirements.txt   # numpy, torch, Pillow, modelscope

Restart, then download the model from ModelScope into ComfyUI/models/LLM/gemma-4-12b-it/. It won't fetch itself - an empty folder just means a load error on the first run.

Where people get burned

Same walls as the whole pack, so don't repeat the discovery. Full-precision 12B is roughly 24GB in bf16, which is why the community's common Gemma 4 setup is a 4-bit Ollama build instead - if your card is 16GB, this node will fight the sampler for memory, and keep_model_loaded off is your only lever short of uninstalling. Also, video frames don't decode themselves: whatever feeds video_frames has to hand over an IMAGE batch, so a separate video-to-frames step sits upstream. It's a new, small, low-profile pack - open code, worth a skim before first run, same as anything in this category.