Enable MonarchAttention (self-attn)

What it actually does

Monarch Attention is the subquadratic attention idea from the Stanford/Together research line (arXiv 2405.19263, "Monarch Attention: A Subquadratic Attention Mechanism for Generative Language Models"). Instead of computing the exact attention matrix, it approximates it with a Monarch matrix - a product of small block-diagonal factors - which you can apply to a sequence in better-than-quadratic time. The pitch, straight from the papers people pass around on r/comfyui: kernel speedups of roughly 1.4–11.8x on long sequences - which is why it keeps showing up in "wouldn't this speed up video generation?" threads.

This node is the ComfyUI hook for that idea. No API calls, no model files, no global patching. It swaps the attention implementation for the self-attention on the MODEL branch you wire it into, and nothing else.

How it works (the honest version)

Under the hood it clones your model and sets optimized_attention_override in model_options["transformer_options"] - the same hook SageAttention and FlashAttention nodes use, and the reason only the model that flows through the node is affected.

Then comes the gate. The override only fires for square self-attention where Q, K, and V all have the same sequence length. Cross-attention - the text conditioning in SDXL - never touches it, and neither does anything outside your min_seq_len/max_seq_len window (defaults 256–4096). For still images, sequences are short enough that subquadratic buys you little. Video is the interesting case: Wan and Hunyuan temporal self-attention grows with frame count - exactly where a cheaper kernel might pay off (the README's "WAN branch" example is no accident).

Two design details worth knowing. First, if the model already carries an override (say SageAttention), this node chains it as a fallback instead of clobbering it - and when you disable MonarchAttention later, the old override comes back. Second, remember this is an approximation, not exact attention. It can change output quality, and nobody has published a ComfyUI benchmark; the speedup figures come from language-model kernels, not diffusion.

The inputs that matter

Eleven inputs, but most are research knobs you can leave alone. The ones you'll actually touch:

• model - your diffusion model. Place this node on the MODEL line before the sampler.
• enable (true) - set it false and the node behaves like the Disable node, removing the override.
• impl (auto) - auto picks triton if the vendored library has it, otherwise torch. If the triton path crashes, pin it to torch.
• block_size (128) and num_steps (2) - the Monarch block size and the number of factors. More steps gets you closer to exact attention at more cost; if you see quality drift, raise num_steps before you give up.
• min_seq_len / max_seq_len (256 / 4096) - the sequence-length window the override will fire within. If the node is silently not engaging, check these first.
• strict (true) - when true, a rejected call raises an error. Set it false and it falls back to the previous attention path instead. For a first test, false is friendlier.

Outputs: model (feed it into your sampler) and status (a STRING recording what got set - handy to eyeball, or pipe into a text node).

Install

The pack itself installs like any custom node. ComfyUI Manager - search "ComfyUI_Monarch_Attention" - or:

cd ComfyUI/custom_nodes
git clone https://github.com/xmarre/ComfyUI_Monarch_Attention

Restart ComfyUI. There's no requirements.txt, no pip dependencies beyond torch (triton is optional, and impl=auto falls back gracefully), and no model to download. GPL-3.0 licensed, so keep that in mind if you're building something to distribute.

Here's the gotcha the README glides past: the actual MonarchAttention implementation is not in the repo. The third_party/ folder ships empty. On your first run, the node raises an ImportError explaining that the vendored ma/ package is missing, and asks you to place the MonarchAttention repo's ma/ folder into third_party/monarch_attention/ (or one of the -attention / -main path variants). Budget a second clone and a folder placement before this will run.

Troubleshooting

• ImportError about a "vendored dependency missing" - the step above. Get the MonarchAttention implementation repo and drop the folder that contains ma/ into third_party/.
• Node loads but speed doesn't change - watch the console. With print_debug on (the default), it prints [MonarchAttention] USED when the override fires and [MonarchAttention] REJECT with a reason when it doesn't. If you only ever see REJECT, your sequence length is outside min/max, or you're feeding it cross-attention.
• Errors mid-generation - either drop strict to false so it falls back instead of raising, or switch impl to torch.
• Want out? - the Disable node removes the override and restores whatever came before, or just set enable to false.

Real talk: this is a young, one-person experiment. Its only community signal is a Reddit thread where someone asked "will Monarch attention work in ComfyUI?" and a commenter answered "someone implemented it, haven't tested it yet." Treat it as a curiosity to A/B against SageAttention on a video workflow - not a drop-in speed multiplier.