Nodes/DiffusionGemma Prompt Builder/DiffusionGemma Grounding Guard Settings
ComfyUI Node

DiffusionGemma Grounding Guard Settings

Stops DiffusionGemma from claiming it saw things it can't verify

By exportAnything·Created 2 months ago·Updated a day ago· 44
DiffusionGemma Grounding Guard Settings
    • grounding_guard_config
    • config_json
    modeaudit
    retry_on_uncertaintrue
    sampling_profilecheckpoint_defaults
    seed0
    save_detailed_tracefalse
    trace_subfolderdiffusiongemma_grounding
    external_evidence_json
    evidence_token_budgetauto

    Vision-language models hallucinate reference details the same way they hallucinate anything else: DiffusionGemma "sees" your source image and writes an LTX or H3 prompt claiming a face, a wardrobe, a camera move. Usually that's a feature. When the claim is false - the girl was never wearing red, the video never had that dolly-in - it becomes a bug that lands directly in your render. DiffusionGemma Grounding Guard Settings is the pack's answer: a configuration node that makes the prompt author back up its visual claims with evidence before a prompt is ever allowed downstream.

    It's a settings node, not a gate. It doesn't validate anything itself. It emits a DG_GROUNDING_GUARD_CONFIG bundle that you feed into the CoT Generator's grounding_guard_config input, and the CoT Generator does the guarded generation work. Leave the input disconnected and the CoT Generator falls back to a non-blocking audit mode. The official docs are in the pack's docs/GROUNDING_GUARD.md, which spells out the mode contract, evidence schema, and trace rules.

    The three modes

    • off - exact legacy behavior. DiffusionGemma writes prompts with no evidence checking. Pick this when you just want fast, unverified prompt authoring and don't care about the guardrails.
    • audit - evidence is collected and reported as diagnostics, but nothing blocks. Good for seeing how much of what DiffusionGemma claims is actually verifiable before you commit to strict.
    • strict - the compliance mode for the flagship workflows. Unverifiable visual claims block the prompt. This is what makes a "prompt claims X, evidence can't confirm X" situation fail closed instead of silently reaching your sampler.

    Settings that actually matter

    • seed - a scoped 64-bit seed, and here's the subtle bit: in audit/strict mode it also selects the Director variant. So refresh with a fixed seed intentionally reproduces the same prompt. Change the seed (or the after-generate policy) when you want a genuinely new variant.
    • retry_on_uncertain - strict mode may make one extra factual evidence attempt; audit mode may retry once when a packet is malformed or salvaged.
    • sampling_profile - checkpoint_defaults normally; full_48_diagnostic runs the native 48-step schedule with adaptive stopping disabled when you're debugging.
    • evidence_token_budget - auto (768 tokens, 1024 on retry) or an explicit 768/1024/1280. Higher values take longer but don't weaken validation.
    • save_detailed_trace / trace_subfolder - opt into JSON-only evidence traces beneath ComfyUI/output/diffusiongemma_grounding. Media and logits are never copied.
    • external_evidence_json - optional typed dg-external-evidence/1 claims you can paste from local OCR, detectors, pose/tracking tools, SigLIP, or manual review. This is how you hand the guard ground truth it couldn't derive itself.

    How it sits in the graph

    Context Hub → Target Profile → CoT Generator, with the guard Settings feeding the CoT Generator. strict demands that every connected Picture and Video role in an H3 Ref2VA manifest carry a host-owned [dg:...] annotation (like [dg:identity,appearance]) so role validation is deterministic rather than inferred. Audit mode may infer category vocabulary from unannotated prose for compatibility diagnostics. Annotations are stripped before compilation and never reach the H3 prompt.

    Installing it

    It's part of the exportAnything/ComfyUI-DiffusionGemmaPromptBuilder pack:

    cd ComfyUI/custom_nodes
    git clone https://github.com/exportAnything/ComfyUI-DiffusionGemmaPromptBuilder
    

    or search DiffusionGemma Prompt Builder in ComfyUI Manager and restart. Requirements are the pack-wide set (transformers>=5.12.1, accelerate, comfy-kitchen, jsonschema, safetensors, …); the heavy part remains the ~26B NVFP4 model in ComfyUI/models/LLM/diffusiongemma-26B-A4B-it-NVFP4 that the CoT Generator actually runs.

    Common issues

    • Blocked runs you can't explain. The gate downstream reports "Grounding Guard blocked generation" with reasons; read the diagnostic text in the error - it lists the specific unverified claims. This usually means a reference image is genuinely too ambiguous, or a role lacks its [dg:...] annotation.
    • The guard's word isn't law. It's a structural evidence check - for the flagship advertisement workflow the README is explicit that a bare pass without evidence is downgraded to not_measured. Strict grounding prevents unverified claims from reaching the splitter and gate, but it isn't a substitute for human review of identity, product, and copy.
    • Reproducibility gotchas. Detailed grounding traces deliberately bypass both the disk cache and the in-memory cache so their side effects never get skipped; and remember a fixed guard seed plus reuse gives you the identical prompt every queue - change the seed for variation.
    Categoryprompt/diffusiongemma

    Inputs (8)

    NameTypeDefaultDescription
    modeCOMBOaudit3 options: off, audit, strict
    retry_on_uncertainBOOLEANtrueStrict mode may make one additional factual evidence attempt. Audit mode may make one fresh combined retry when an LTX or H3 packet is malformed, salvaged, or contains an invalid grounding ledger.
    sampling_profileCOMBOcheckpoint_defaultsNative 48-step 0.8-to-0.4 DiffusionGemma schedule; diagnostic disables adaptive stopping.
    seedINT00–18446744073709550000Scoped unsigned 64-bit seed; global CPU/CUDA RNG state is restored after each call. In audit/strict mode this also selects the Director variant, so refresh with a fixed seed intentionally reproduces the same result. Use randomize/increment when you want a new prompt variant.
    save_detailed_traceBOOLEANfalseOpt in to JSON-only evidence traces beneath ComfyUI/output; media and logits are never copied.
    trace_subfolderSTRINGdiffusiongemma_groundingRelative subfolder beneath ComfyUI/output.
    external_evidence_jsonoptSTRINGOptional dg-external-evidence/1 typed claims from local OCR, detectors, pose/tracking/motion, SigLIP, or manual review.
    evidence_token_budgetoptCOMBOautoStrict evidence-only output budget. Auto uses 768 tokens initially and 1024 on retry; an explicit value applies independently to every evidence attempt. Higher values take longer but do not weaken validation. Ignored in off/audit modes.

    Outputs (2)

    NameTypeDescription
    grounding_guard_configDG_GROUNDING_GUARD_CONFIG
    config_jsonSTRING