GemmaRun

If you've ever wanted to generate a prompt with a real language model instead of hand-typing it, this is the node that did it first inside ComfyUI. GemmaRun takes a text prompt, runs it through Google's Gemma model loaded in RAM, and hands you back a STRING with the model's reply. No API calls, no keys at runtime, no network - the whole thing runs on your machine, as long as your machine can hold a Gemma checkpoint.

It's worth being honest about what you're getting, though. This pack shipped in February 2024, literally days after Google released Gemma, and it's a thin wrapper around Google's own reference implementation (gemma/model.py in the repo). The author posted it to r/comfyui the same day and got zero comments; the repo has one commit. It's a time capsule, not a maintained project. If you just want a text LLM in your graph today, the GGUF-based packs are more capable. But GemmaRun still works, it's small, and it's a genuinely good way to understand what an LLM node is doing under the hood.

How it works

GemmaRun never loads anything itself. It takes a GemmaModel from the pack's other node, GemmaLoader, which reads a tokenizer.model plus a gemma-<variant>.ckpt checkpoint from your local kagglehub cache, builds the model, and moves it to the device. GemmaRun then does the inference: it seeds Python's random, numpy, and torch, and calls model.generate().

The generation settings are worth knowing because they're not inputs. The author hardcoded them: 60 output tokens max, temperature 0.95, top-p 1.0, top-k 100 - the defaults straight out of Google's reference script. That means whatever you ask for, you get at most 60 new tokens, and there's no knob to lengthen it without editing nodes.py. For short jobs - a caption, a rewritten prompt, a title - that's fine. For anything essay-length, it isn't.

The inputs and outputs that matter

Four required inputs, and only two you'll actually touch:

• model (GemmaModel) - plug in the output of GemmaLoader. Non-negotiable.
• MACHINE_TYPE - cuda or cpu. Pick whichever matches where you ran the loader.
• prompt (STRING) - the text you send the model.
• seed (INT, default 0) - reproducible output, same as every other seed in ComfyUI.

The single output is result (STRING), which you wire into a Show Text node or any node that takes text - prompt-expansion feeding into a positive prompt, a caption into an Image Caption node, whatever your graph needs.

Installing it

From ComfyUI Manager, search "ComfyUI-Gemma" and install. Or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/chaojie/ComfyUI-Gemma

Then restart ComfyUI and let it run requirements.txt - that pulls kagglehub, sentencepiece, immutabledict, and packaging into your environment. The sentencepiece bit is what tokenizes your prompt into Gemma's tokens.

Here's where the real work starts, and where most people bounce off. The weights aren't bundled - you download them from Kaggle, after creating an account and accepting Gemma's terms. Then export your credentials and let kagglehub do the download:

export KAGGLE_USERNAME=your_username
export KAGGLE_KEY=your_api_key   # from Kaggle > Settings > API
python -c "import kagglehub; print(kagglehub.model_download('google/gemma/pyTorch/7b-it-quant'))"

Variants are 2b, 2b-it, 7b, 7b-it, 7b-quant, 7b-it-quant - the -it ones are the instruction-tuned models, which is what you want for prompting tasks. The quant variants are int8 and roughly half the size (7b quant ~7GB vs ~14GB for bf16).

Common issues

• The default weights_dir points at the author's machine. GemmaLoader's path field defaults to /home/admin/.cache/kagglehub/... - that folder does not exist on your box. Run the kagglehub download yourself, copy the printed path, and paste it in.
• MACHINE_TYPE mismatch. Both nodes take it, and they should agree. Load the model on cuda but set GemmaRun to cpu, and you'll get a device-mismatch error when the KV caches are allocated. Keep them consistent.
• 60 tokens. It's not a bug, but it surprises people. The reply is hardcoded short; edit output_len=60 in nodes.py if you must.
• VRAM. 7b bf16 is ~14GB on top of whatever ComfyUI already holds - real cards only. The 2b fits on modest GPUs and will crawl but run on CPU.
• Security. Custom nodes are arbitrary Python with your user-level access, and this one reads your Kaggle credentials from the environment. That's normal for the ecosystem, but it's why you don't install unknown nodes blindly - the LLMVISION malware incident made that point loudly.
• Licensing. The code is Apache 2.0, but the weights ship under Google's Gemma Terms of Use, not Apache. Fine for personal use and most commercial work, but skim it before you build a product on it.

One last take: if you just want a quick, local "write me a prompt" tool and you have the VRAM, this is a charmingly simple way to get it. If you need length, speed, or a maintained project, look at the GGUF route instead. GemmaRun is the historical footnote that still runs - and sometimes that's exactly what you need.