DiffusionGemma Sampler
Watch text anneal out of pure noise — DiffusionGemma's sampler as an instrument
- model
- text
- canvas_state
- canvas_trace
- frames
- images
- run_config
Here's the difference between this sampler and every other sampler in ComfyUI: it isn't denoising pixels, it's denoising text. DiffusionGemma doesn't write an answer word by word. It starts with a canvas of 256 random tokens drawn from the whole multilingual vocabulary - maximum entropy, zero meaning - and then anneals. A temperature schedule cools across the run; at each step the positions the model is most confident about freeze into place while the rest get re-noised and tried again. The confident tokens crystallize first, the uncertain ones settle last, and a coherent answer precipitates out of the noise. That's what DGemmaSampler runs, and it hands you every scrap of the process on separate sockets so you can actually watch it.
If you just want text out, this node is overkill. If you want to see where meaning locked in - and sometimes watch the model anneal confidently into a wrong answer it can't climb out of - it's the whole point of the pack.
The knobs that matter
You get model and prompt up top, then a wall of parameters. A few you'll actually set:
seed- plain integer for the canvas init/renoise generator. Leave unset for a nondeterministic run.num_inference_steps(48) - the step budget. More steps, slower anneal.entropy_bound(0.1) - the per-step acceptance budget, in nats (natural-log entropy), not bits. A position commits once its acceptance entropy clears this. For scale: the uniform noise draw is ~12.48 nats, so 0.1 is a tight budget.gen_length(256) - canvas length in tokens, processed in blocks of 256. Go bigger and you pay proportionally more block passes.thinking- experimental toggle that injects the<|think|>control token. Can spend the whole canvas thinking and return nothing; the readout flags it.
The fiddly trap: t_min/t_max look like schedule positions but are temperatures - the softmax divisor. And knob response isn't a smooth dial; block autoregression makes output respond discontinuously, in plateaus and cliffs. Don't fight it, just know a tiny entropy_bound nudge can change nothing, then everything.
What you get back
Six outputs, and they compose into a real instrument chain:
text- the clean answer; the thought channel is excised at the id level, never leaked.canvas_state- the honesty readout:converged,turn_closed,answer_tokens,committed_fraction. The key one: if you're asking "did this run finish?", readturn_closed, notconverged- adaptive stopping can legitimately end withconverged=Falseon a clean run.canvas_trace- per-step analysis data. Wire intoDGemmaTraceorDGemmaTokenTrace.frames- one decoded string per captured step, the in-graph flipbook from noise to text.images- those same frames as a single batched IMAGE. Because it's a standard batch (not a per-frame list), it plugs straight into VideoHelperSuite'sVideo CombineorSaveAnimatedWEBPfor a shareable GIF of the model thinking. This is the output people actually post.run_config- the run's header bundle (seed, knobs, model identity). Feed it, plus the other outputs, intoDGemmaRunLogWriterfor a kept record.
Install and gotchas
Install the pack once via Manager (search "ComfyUI-DiffusionGemma") or clone, and it needs transformers==5.13.0 + diffusers>=0.39.0. The real cost is upstream: the loader's ~54 GB bf16 model, 48 GB VRAM comfortably, 24 GB squeezed with ComfyUI's offload and a lot of system RAM. Steps are slow (~2.3 s/step on a 48 GB card) because of offload.
Watch for two known behaviors: thinking=true can burn the whole canvas and return an empty answer (it tells you: turn_closed=False, answer_tokens=0), and the telemetry measures when positions settled - not whether they were diffusion-computed versus recalled from a memorized prior. Read the commit curve as "when did this settle," not "this was genuinely computed here."
Inputs (10)
| Name | Type | Default | Description |
|---|---|---|---|
| model | DGEMMA_MODEL | Loaded DiffusionGemma model (from DGemmaLoader). | |
| prompt | STRING | The user turn to generate a response to. | |
| seed | INT | 00–18446744073709550000 | RNG seed for the generator driving canvas initialization/renoise (a plain integer, not unit-bearing). Omit/leave unset for a nondeterministic run. |
| num_inference_steps | INT | 481–1024 | Requested denoising step budget N (a plain count, not a physical unit) — the schedule-position denominator: t = (N - step_idx)/N. |
| t_min | FLOAT | 0.400–1 | Cold end of the temperature anneal (dimensionless, applied as the divisor T in softmax(z/T)). Despite the lowercase-t name this is a TEMPERATURE, not a schedule position — t bottoms out at 1/num_inference_steps, so t_min itself is a virtual endpoint no step actually reaches. T = t_min + (t_max - t_min) * t, t decreasing 1 -> 1/num_inference_steps across the run. |
| t_max | FLOAT | 0.800–1 | Hot end of the temperature anneal (dimensionless, same softmax(z/T) divisor as t_min) — the temperature applied at the very first step, where the schedule position t == 1. |
| entropy_bound | FLOAT | 0.1000–1 | Per-step joint acceptance budget, in NATS (natural-log entropy, matching torch.distributions.Categorical.entropy() — not bits). A position commits this step only once its acceptance entropy clears this bound. Default 0.1 nats; for scale, the uniform-vocab noise draw's 18 bits/position is ~12.48 nats. |
| confidence | FLOAT | 0.0050–1 | Early-stop threshold: a dimensionless probability the pipeline's adaptive-stop check compares a candidate's confidence against. |
| gen_length | INT | 2561–8192 | Canvas length in tokens (a token count) — how many positions the denoising loop allocates for the generated turn. This is split into blocks of DEFAULT_GEN_LENGTH (256) tokens each: gen_length is processed as ceil(gen_length / 256) blocks, each denoised over num_inference_steps substeps (e.g. gen_length 1024 -> 4 blocks, each denoised over num_inference_steps steps). Larger gen_length costs proportionally more block passes. |
| thinking | BOOLEAN | false | EXPERIMENTAL boolean toggle: injects the <|think|> control token via a system turn. Structurally one token short of native enable_thinking=True (the chat template's `| trim` eats the newline after <|think|>) — see run_diffusion's own docstring for the honest gap. Behavioral impact unverified pending an E2E thinking-mode run on real weights. |
Outputs (6)
| Name | Type | Description |
|---|---|---|
| text | STRING | — |
| canvas_state | DGEMMA_CANVAS_STATE | — |
| canvas_trace | DGEMMA_CANVAS_TRACE | — |
| frames | STRING | — |
| images | IMAGE | — |
| run_config | DGEMMA_RUN_CONFIG | — |