Attention Map Bending (Experimental)

Regional prompting, latent couple, attention couple - the ComfyUI ecosystem has been manipulating attention for years, mostly by masking it so the girl on the left keeps her red hair. Attention Map Bending is the same idea taken down a level and made blunt: take the cross-attention map between a video DiT's tokens and your prompt, and transform it with any bending op from this pack. Rotate it, scale it, blur it, translate it, flip it.

The result is video that slides sideways, textures that crawl, subjects that drift downward through the frame. It's the most obviously "creative" thing in the pack and the most experimental, in the literal sense that the author says it's only been validated against tiny random-weight WAN models in the test suite, not real weights.

What it's doing

For the cross-attention cases it does the honest, expensive thing: compute the map explicitly. The softmax of Q·Kᵀ/√d between the video tokens and the text tokens is materialised, each prompt token's column is laid out on the latent (frames, height, width) grid, every frame is bent by your module, and then - depending on renormalize - the columns are rescaled so the attention still means something. Then the bent map goes back into the multiplication with V.

Self-attention is cheaper, because materialising an L×L map for 30k+ tokens is not happening:

• self_query bends where each position's result lands - the attention output is laid out on the grid and bent, which for a linear op is mathematically the same as bending the map.
• self_key bends where each position reads from: the values get moved on the grid before attention, so content is pulled in from a different location while the layout stays put.

Under the hood it hangs off ComfyUI's optimized_attention_override path, which is why it only works on models whose attention calls route through transformer options. Practical translation: WAN 2.1/2.2, not SDXL, not Flux.

Inputs that matter

• attention - cross_text (the default: video-to-prompt), cross_image (video-to-CLIP-image, for WAN 2.1 I2V), self_query, self_key.
• blocks - 13-18 is the middle of WAN 2.1 1.3B's 30 blocks, and roughly 17–24 of 40 on the 14B. The default * bends every block and is the strongest thing you can do, which is not the same as the best.
• tokens - all (every key, padding included), prompt (only the prompt's own tokens), indices like 0-3,7, or actual words like horse, red car if you also wire clip and type the prompt. Note the author's tooltip: single words are much weaker than all. Cross-attention only.
• renormalize - keys (default) makes each video position's attention sum back to 1, i.e. still a valid softmax; per_token_mass keeps each token's total attention; none is raw. Amplifying with a Multiply Scalar module only does anything with none - the node warns you if you pair amplify with keys.
• apply_to - both, cond or uncond. Bending cond alone gets multiplied by the guidance scale, so it's stronger and glitchier.
• strength - blends between the unbent (0) and bent (1) map. Handy for dialling an otherwise violent op back to 20%.
• Optional: steps (0-2 - early steps change layout and coherence, late steps barely change anything), t_start/t_end, heads (12 on 1.3B, 40 on 14B), frames (WAN: one latent frame = four video frames), clip and prompt for word lookups, strict.

Outputs are MODEL and a report string confirming blocks, tokens, heads, frames, steps and which module got used.

Settings that work

From the shipped video workflows and the README's tips: keep scales tiny - a Scale module at 1.04 is a whole effect. A Rotate module at 12–20° across blocks 13-18 is the demo. tokens: all for anything visible. Rotate for early steps (0-2) and a different op for later steps (7-9) as two stacked nodes. On WAN 2.2 14B there are two models - bend each separately, and treat the high-noise one as the layout model.

Cost, and where it hurts

Computing the cross-attention map explicitly means memory and time roughly proportional to tokens × prompt tokens. At 720p and 81 frames on the 14B model that map is around 75k × 512 per head. Bend a few blocks and a few steps at high resolution, not all 40 blocks at every step, unless you enjoy watching VRAM die. Chunking is per sample and per few heads, so it degrades gradually rather than exploding instantly.

Install

Manager → "ComfyUI Model Bending", or:

cd ComfyUI/custom_nodes
git clone https://github.com/abuzreq/ComfyUI-Model-Bending
pip install -r ComfyUI-Model-Bending/requirements.txt

kornia and scikit-learn; restart afterwards. The video workflows in workflows/video/ link their own WAN weights. Console output is tagged [model-bending].

Honest expectations

Video gen is already at 15–40 minutes a clip on a 4090 without speed LoRAs, and this node makes it slower by materialising attention maps the sampler never had to hold. Add that this is a research-grade feature validated on random-weight test models, and the right posture is: try it on a short, low-res clip, on a handful of blocks, before you commit an evening to something beautiful. There is no Reddit thread to crib from either - "model bending" shows up twice in the whole corpus.