Automatic CFG - Attention modifiers

This node doesn't touch the CFG math at all - it's a factory for ATTNMOD objects, the little data packets the rest of this pack uses to rewrite attention. You tell it which transformer block to patch, which attention (self, cross, or both), over what sigma range, and what expression to evaluate inside the attention computation. Out comes an Attention modifier you feed into the Preset Loader's join_global_parameters, the Advanced node's attention modifier inputs, or the Excellent attention / Custom attentions nodes.

It's the same trick that powers a whole family of "perturbed attention guidance" style nodes, except here you write the perturbation yourself. The pack ships a tutorial image in the workflows folder, and the README shows how to reproduce Perturbed Attention Guidance (PAG) this way.

The inputs that matter

• self_attn_mod_eval - the heart of it: a Python expression, evaluated inside the attention function with q, k, v, and extra_options in scope. Something like q (bypass - leave attention untouched), torch.zeros_like(q) (kill it), or the PAG-style q + (q - attention_basic(attnbc, k, v, extra_options['n_heads'])). This is where the actual experiment lives.
• unet_block_id_input / _middle / _output - comma-separated block numbers per UNet section, e.g. 4,5,7,8 input / 0 middle. Leave a section empty to skip it.
• unet_attn - attn1 (self-attention), attn2 (cross-attention), or both. Choosing both emits two modifier entries.
• sigma_start / sigma_end - when during sampling the patch is active, in sigma units. 1000 → 0 is "always".

Optional join_parameters takes another ATTNMOD and appends - so you can chain modifiers through multiple instances of this node into one long list.

Outputs

• Attention modifier (ATTNMOD) - wire into any node in this pack that accepts one.
• Parameters as string - a text summary of what you built, handy for logging or feeding onward.

The warning, said once and loudly

self_attn_mod_eval is evaluated with eval() on every forward pass over the affected layers. It's not sandboxed. Paste something from a stranger and you're running their code on your machine. Writing your own expressions is the whole point of the node - just don't paste.

Also expect to crash things. Wrong block IDs, expressions that reference names that aren't in scope, attnbc used without the helper names the pack defines - the failures are ugly Python tracebacks, not graceful errors. That's fine; it's a brute-force playground. The companion tester node exists precisely because you're meant to try a modifier everywhere and see what sticks. Block numbering follows the UNet's input/middle/output layout and mostly makes sense on SDXL - most of this pack's attention experiments were only validated there.