DiffusionGemma Model Loader (Advanced)

This is the heaviest node in the DiffusionGemma Prompt Builder pack, and the one beginners sweat over the most. It's the door through which a ~26B multimodal LLM - NVIDIA's DiffusionGemma, the diffusiongemma-26B-A4B-it-NVFP4 checkpoint - enters your ComfyUI process. The "Advanced" in the display name isn't marketing: this is the node that decides whether the whole prompt-builder pipeline runs at all.

Quick reality check on what it's not: it doesn't call any API, it doesn't need a key, and it isn't the thing that generates your image or video. DiffusionGemma here is a prompt writer, not a sampler. It takes your text, reference image, and source video, and writes the LTX 2.3 (or Ideogram 4) prompts that the actual generation model consumes. This node just gets that writer loaded and healthy.

How it works

The pack runs the model in-process through Transformers plus a local "Comfy NVFP4 bridge." That bit matters. NVIDIA ships the checkpoint in its own ModelOpt FP4 format, which stock Transformers won't deserialize - so at load time the bridge (comfy-kitchen) converts the FP4 nibble order and weight_scale layout into Comfy's packed NVFP4 expert-matmul layout, all on the fly. No vLLM, no background inference server, no bundled custom tensor file. Keep the model as a normal Hugging Face repo folder and let this node do the translating.

Here's the counterintuitive bit that saves beginners a scare: creating the loader does not load the model. It runs a status probe - path check, processor/class import, an NVFP4 bridge smoke test on one expert layer - and hands back runtime_status_json so you can see what's ready. The real weight load happens lazily on your first pass through the CoT Generator. The loader returning instantly isn't a bug; the VRAM spike is a few nodes downstream.

The inputs that matter

• model_path - point this at the whole HF repo folder you downloaded, e.g. ComfyUI/models/LLM/diffusiongemma-26B-A4B-it-NVFP4. The default is a placeholder path; the README's real location is the models/LLM folder. Get this wrong and you get the user-reported error below.
• quantization - defaults to modelopt_nvfp4 and you should leave it there. none, bitsandbytes_4bit, quanto exist as alternatives, but the whole pack is tuned for the NVFP4 path.
• max_memory_gb - 20.0 GB - safe practical is the default and the sweet spot; 18.0 GB - minimum supported is the floor. Setting it between 0 and 18 is deliberately blocked to avoid a partial CUDA load that can force a ComfyUI restart. 0 GB - full GPU loads everything onto the card if you have room.
• unload_policy - keep_loaded (default) keeps the runtime resident for fast repeat generations; unload_after_run frees VRAM between runs if you're chaining to another heavy model.
• temperature (optional) - sampling temperature, 0.2 conservative, 0.45–0.8 more creative. The CoT Generator can override this per pass, so you can nudge randomness without touching the loader.

Outputs are model_config (the DG_MODEL_CONFIG object the CoT Generator consumes) and runtime_status_json - a string you can dump to a text node when things misbehave.

Install and the model download

Install via ComfyUI Manager (search DiffusionGemma Prompt Builder) or the usual:

cd ComfyUI/custom_nodes
git clone https://github.com/exportAnything/ComfyUI-DiffusionGemmaPromptBuilder

Then restart ComfyUI. requirements.txt pulls in transformers>=5.12.1, accelerate, comfy-kitchen>=0.2.10, jsonschema, safetensors, and tqdm - but note the pack also needs a ComfyUI build that exposes comfy.quant_ops and comfy.ops. The author pins transformers==5.12.1 and comfy-kitchen==0.2.10 in practice.

Then the big one - the model itself is a 26B MoE in FP4, so it's a serious multi-GB download. Place the whole repo folder at:

ComfyUI/models/LLM/diffusiongemma-26B-A4B-it-NVFP4

Before your first real run, verify the environment with the pack's proof gate:

python custom_nodes/ComfyUI-DiffusionGemmaPromptBuilder/proof_gates.py \
  --model-path ComfyUI/models/LLM/diffusiongemma-26B-A4B-it-NVFP4

It should report the processor resolves, DiffusionGemmaForBlockDiffusion imports, CUDA is available, and the NVFP4 bridge smoke test passes.

Common issues

• "Repo id must use alphanumeric chars … 'C:\ComfyUI\models\LLM\…'" - real user report. It means the model path doesn't point at an actual model; you haven't downloaded it yet. Go get the NVFP4 repo.
• "Cannot use apply_chat_template because this processor does not have a chat template" - the processor couldn't resolve from a proper model folder. Place the whole HF repo (config, processor files, safetensors), not just the weight files.
• VRAM reality. NVFP4 is tuned for Blackwell (CUDA capability 12.0). On a 3090 or other Ampere card it processes the slow way and can OOM - the author is blunt about this. 18 GB is the tested floor; the full packaged workflow ran at 24–26 GB VRAM for them. If you're under that, this pack isn't yours yet.