⚙️ GRAG Advanced Sampler v3.0

Here's the thing nobody tells you about this pack: the two controllers don't actually do anything. They just write notes on the conditioning. GRAG_Advanced_Sampler is the node that reads those notes and changes the image. It looks like a reskinned KSampler, and that's exactly the point - it's a KSampler replacement with a trick inside.

GRAG (Group-Relative Attention Guidance) is a training-free image editing technique that reweights the attention keys inside a diffusion model instead of fine-tuning the weights. The formula is deceptively small: k̂ = λ × k_mean + δ × (k − k_mean), where you pull each key token's deviation from the group mean, then push that deviation up or down with δ while λ dials the whole group. The pack is built on the GRAG-Image-Editing paper (arXiv 2510.24657). Because it's training-free, it's a natural fit for the Qwen-Image editing ecosystem - ComfyUI loads Qwen-Image natively, and this sampler targets exactly the attention blocks in that architecture.

How it actually works

When you run it, the sampler does four things:

1. Reads the GRAG metadata out of the positive conditioning (the grag_enabled, grag_lambda, grag_delta keys that the controllers embedded).
2. Clones the model so your original isn't touched.
3. Finds every Qwen-style Attention module and monkey-patches its forward pass: compute QKV, apply RoPE, then reweight the keys with k̂ = λ·k_mean + δ·(k−k_mean) right before attention is computed.
4. Runs ComfyUI's standard sampling, then - critically - restores the original forward methods in a finally block. That restoration is the "v2.2.1 contamination fix," and it matters: a monkey-patch left behind would leak GRAG into every other sampler in your graph. It's the kind of bug that drives people insane, and this pack specifically fixed it.

One honest caveat from the source: the patch only matches Qwen-style attention modules (class Attention with to_q, add_q_proj, norm_q). The README claims auto-detection for "Qwen, Flux, SD3," but the code is really Qwen-specific. On a non-Qwen model you'll get "Patched 0 Attention layers" and a silent fallback to vanilla sampling.

The inputs that matter

It's a straight KSampler clone: model, positive, negative, latent_image, seed, steps, cfg, sampler_name, scheduler, denoise. Wire it exactly like you would a KSampler - same model, same latent. Three things worth knowing:

• positive must come from a GRAG controller (Simple or Unified). Feed it a plain conditioning and the sampler just does normal sampling with zero GRAG.
• denoise is standard (1.0 = full denoise), handy for img2img-style edits with a VAE-encoded input.
• debug_mode is the one extra knob. Flip it and the console prints every patched layer plus performance metrics - the easiest way to confirm GRAG is actually running.

Output is a single samples (LATENT), which goes straight into VAE Decode like any other sampler.

Installing it

Same story as the whole pack - install it via ComfyUI Manager (search "GRAG" or "ArchAi3d"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/amir84ferdos/ComfyUI-GRAG-ArchAi3D
# optional: full preset library
pip install PyYAML

No model downloads, no torch-gpu dependency shims, no gguf files. PyYAML is the only optional dependency and it's for presets, not for sampling. This is refreshingly lightweight next to the packs that drag in half of PyPI.

Troubleshooting

If you flip everything on and see no change, walk the checklist: enable_grag is True on the controller, the conditioning is wired through that controller, the preset has λ≠1.0 or δ≠1.0, and - the big one - your model is Qwen-based (Qwen-Image, Qwen-Image-Edit). Check the console for "Patched XX Attention layers"; if it says zero, GRAG isn't reaching any attention blocks.

The other classic gotcha is ComfyUI itself. The v3.0.1 release was a compatibility fix for a ComfyUI rewrite of Qwen attention internals (the tensor format moved to BHND, and the RoPE helper moved to comfy.ldm.flux.math). So if this node suddenly throws dimension errors after a ComfyUI update, that's the known failure mode - update the pack, and if it's still broken, that's a real GitHub issue to file, not you doing something wrong.