[prototype] Coarse Global Attention (not adopted)

The problem it was trying to solve

Self-attention on a 1792×2944 latent is quadratic in token count and it's the thing that makes big images fall over on a small card. Windowed attention is the classic answer - split the token grid into blocks, have each block attend only within itself, and the cost drops from n² to n·window². The catch is that windows forget each other, so the left side of the image has no idea what the right side is doing.

Coarse Global Attention is the pack's attempt at keeping some global context: each window's tokens attend to their own window at full resolution plus the whole image pooled 2×2. Cheap global summary, expensive local detail. It's a sensible design and it shows up in various forms in the long-context literature.

It didn't work here. The author's note in enhancers/prototypes.py is two words long and unsparing: "Rejected: slower, seams." Slower, because the pooled global branch adds work that windowing was supposed to save; seams, because windows with a coarse global hint still don't agree across their boundaries. Measured and not adopted. The node ships with mode at off.

How it works

When enabled, it wraps the model's self-attention calls. Above a token threshold, it lays a windows × windows grid over the token lattice, has each window attend within itself, and then has everything attend to a pooled version of the whole image. The pooling factor is pool, and kernel chooses whether the global branch uses whatever attention function is currently active - including SageAttention, if you have it on - or PyTorch's SDPA with a modified bias.

That last option exists for benchmarking rather than for use. sdpa_bias adds + log(pool²) to the pooled keys, which is the correction you'd want if you were being rigorous about attention mass, and it's slower. The author kept it around to check whether the seams were an implementation artifact of the fast path. They weren't.

Inputs and outputs

• model - the model to patch.
• mode - off (default) or window_global. Off returns the model unchanged.
• windows (4, range 1–16) - blocks per axis.
• pool (2, range 2–8) - the pooling factor for the global branch.
• min_tokens (12000) - only self-attention with at least this many tokens is touched.
• kernel - attention (use the current attention function) or sdpa_bias (PyTorch SDPA with the log-correction bias; slower, for quality comparison).

One output, MODEL.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/nyanmi-1828/ComfyUI-WeightFerry

or Manager → WeightFerry → install → restart. No dependencies, no model files. The pack needs NVIDIA and ComfyUI 0.38.0+. The node sits under WeightFerry/prototype (not adopted), prefixed [prototype] in the menu.

Where it goes wrong

The default min_tokens means it's inert at normal sizes. This is the trap worth knowing. The tooltip's own scale note says 896×1472 is 5152 tokens - well under the 12000 default. So on a typical Anima generation this node does nothing at all, even with mode set to window_global. If you're trying to reproduce the author's "slower, seams" result, you have to drop min_tokens below your actual token count. Otherwise you'll conclude it's harmless and fast, which it is, because it never ran.

Expecting it to save VRAM. Windowed attention saves compute, not weights. If you're OOMing on an 8GB card, this is the wrong lever entirely - you want the DiT streaming node. Different problem, different mechanism.

Reading "prototype" as "disabled". It isn't a stub. Set mode and lower min_tokens and it will patch your attention and change your image. The [prototype] prefix and the off default are there so you can't do that by accident.

Chasing the seams with windows. Smaller windows make the seams worse, not better - there are more boundaries and each window sees less. If you're genuinely interested in this approach, windows 2 with pool 2 and the sdpa_bias kernel is the configuration that gives the idea its best shot, and the author already reports it still losing. That's the useful takeaway: the idea was fairly tested, and tiling with proper overlap (Ultimate SD Upscale) beat it.

Whatever you do, don't stack it with the streaming node and expect a compound win. Both change how the model runs; only one of them is bit-exact and only one of them is faster. If you want the low-VRAM path, use DiT Layer Streaming and leave this off.