Apply SpargeAttn

Straight talk up front: this is the most experimental node in the pack, and it says so itself. SpargeAttn (from thu-ml, the same team behind SageAttention) adds sparsity on top of SageAttention's quantized attention - skipping attention computation between tokens that don't meaningfully interact, not just computing it in lower precision. On paper that's a bigger speedup than SageAttention alone. In practice it needs per-model hyperparameter tuning before it's usable at all, and even people who know what they're doing have reported it not being worth the trouble.

What it is and why you'd reach for it (and when you shouldn't)

SpargeAttn made noise on release with claims of up to a 1.8x additional speedup on video models with "no quality loss" - but that headline comes with a huge asterisk: those numbers require hyperparameters tuned specifically to your model. Kijai, who maintains the widely-used Wan wrapper and tests new acceleration techniques the moment they land, tried it early: on a 1.3B Wan model, tuning took about five hours on a 4090, the resulting parameters gave only a 10% speedup over plain SageAttention, and quality took "an immense" hit. His read, echoed by other testers, was that the tuning either needs much more work or doesn't transfer well to video the way it does on the still-image benchmarks it was built for. He also noted tuned parameters are model-size specific: a 1.3B tune doesn't carry over to a 14B model.

The honest recommendation: most people should skip this node. Reach for it only if you're already squeezing every other trick in this pack (SageAttention, caching, torch.compile) and are willing to spend real GPU-hours on a tuning run for marginal, unproven gains - and treat "no quality loss" as a claim to verify yourself, not to assume.

How it works

Two-phase workflow, both built into this one node. In tuning mode, you run a batch of generations while the node measures how sparse the attention pattern can get before quality suffers, and learns hyperparameters specific to your model. Once tuned, you save those parameters (with the companion Save Finetuned SpargeAttn Hyperparams node) and load them back in for normal generation, where the node applies sparse attention using what it learned instead of re-tuning every time.

The inputs and outputs that matter

• model - your loaded diffusion model.
• enable_tuning_mode (boolean, default false) - flips the node from "apply sparse attention" to "learn hyperparameters." The README's own workflow: generate 50-step 512x512 images across 10 different prompts with this on, which is described as "very time-consuming" - budget hours, not minutes.
• l1 (default 0.06) and pv_l1 (default 0.065) - sparsity tuning parameters, now user-adjustable per the author's latest changes. Higher generally means more aggressive sparsity (more speed, more risk).
• parallel_tuning (boolean, default false) - spreads a tuning run across multiple GPUs to speed it up. If you enable it, you need to start ComfyUI with --disable-cuda-malloc or it won't behave.
• tuned_hyperparams - the dropdown to load a saved hyperparameter file once you have one, instead of retuning every session.
• skip_DoubleStreamBlocks / skip_SingleStreamBlocks (defaults "3,6,8,12" / "") - the same block-skip mechanism as the pack's other Flux nodes, mainly there so SpargeAttn can coexist with TeaCache/FBCache on the same blocks.

Output is a patched MODEL.

How to install it

The node pack itself installs normally through ComfyUI Manager (search "ComfyUI-Lightning") or git clone https://github.com/shenduldh/ComfyUI-Lightning into custom_nodes. SpargeAttn the library is a separate install, not bundled:

git clone https://github.com/thu-ml/SpargeAttn.git
cd ./SpargeAttn
pip install -e .

It builds on top of SageAttention, so get that working first (see the Apply SageAttention article for the Triton install pain that entails) - SpargeAttn without a working SageAttention underneath it isn't going anywhere.

Common issues & troubleshooting

Install fails. SpargeAttn inherits every SageAttention dependency headache (Triton, matching CUDA/torch versions) plus its own build step. If pip install -e . fails, the original repo's issue tracker is the more useful place to look than general Flux troubleshooting threads.

Tuned it and quality got worse, not better. This isn't unique to you - it's the documented, reported outcome from people with deep experience tuning attention accelerators. Consider whether the marginal speedup is worth it, and don't assume a bad first tune means you did something wrong.

Tuned on one model size, applied to another, results are bad. Hyperparameters are model-size specific. A tune done on a smaller variant will not transfer cleanly to a larger one - retune per model, which multiplies the time cost.

Tuning is unbearably slow. That's expected and stated in the README itself. parallel_tuning with multiple GPUs is the only lever the node gives you to cut that down, and it requires the --disable-cuda-malloc launch flag to work correctly.