H3 SLA Attention

The missing kernel behind the lightx2v turbo LoRA

By PlagueKind·Created 4 months ago·Updated 5 days ago· 165

H3 SLA Attention

model

MODEL

◄sparsity_ratio0.80►

◄block_size32►

◄min_seq_len12228►

◄dense_last_steps0►

◄protect_audiofalse►

◄enabledtrue►

◄dense_steps1►

◄dense_backendcomfy_kitchen►

◄disable_fp16_accumtrue►

◄stabilize_motionfalse►

◄reference_protectionOff►

◄tail_correctionfalse►

◄use_int8_qktrue►

◄enginecomfy_kitchen►

If you've dropped the lightx2v SLA turbo LoRA into a MiniMax-H3 workflow and wondered why it didn't make anything faster - that's not you doing it wrong. The LoRA only makes the model tolerate sparse attention; the sparse attention itself is a piece of the inference path that plain ComfyUI never had. H3SLAAttention is that missing piece, shipped as a node.

How it works

Block-sparse attention is the trick: instead of attending every token to every token (quadratic, expensive), each query block is scored against every key block with one small pooled matmul, and only the top 1 - sparsity_ratio fraction actually gets attended. Nothing here is trained and nothing is downloaded - the sparsity is decided at runtime from the query and key tensors. That's the whole point of the pairing: the lightx2v SLA turbo LoRA was distilled against this exact inference path, so its job is to make the model behave well under the sparsity, not to provide it. The node wires itself in through transformer_options["optimized_attention_override"] at H3's one attention call site - not the legacy set_model_attn1_patch hook, which is the SD-UNet cross-attention path H3 never consults (a patch installed there reports success and silently does nothing). The kernel itself is Triton.

The author measured end-to-end on a 5090 at 768p/15s with the SLA turbo LoRA: ~44 s/it dense, ~31 s/it at 0.85 sparsity, ~25 s/it at 0.90 - call it 1.4–1.75x, with no extra VRAM. Attention is only ~30s of that 44s step, so the Amdahl ceiling is around 3.17x no matter how fast attention gets; the widely-quoted 2.5x is an eight-GPU number. Long sequences benefit most - the gain grows with resolution and duration.

The inputs that matter

model - MODEL, after the LoRA loader(s), last before the sampler.
sparsity_ratio - fraction of key blocks skipped. 0.85 is what lightx2v ships and what the LoRA was distilled against; 0.90 is what this author validated, ~15% faster. Here's the trap: below about 0.60 this kernel is slower than dense attention, so a low value is a loss, not a safe fallback. 0.0 disables sparsity without removing the node.
block_size - 64 or 128. This matters far more for audio than video: H3 packs audio at 80 rows per second, so a 128-row block forces 1.6 seconds of audio down a single attention pattern. Speech came out robotic at 128 and clean at 64 for ~2% more time. Use 128 only if you generate without meaningful audio.

The optional inputs are the interesting ones. min_seq_len (default 8192) keeps short sequences dense - it guards the short text-refiner attention, which must never be sparsified, and short clips where block selection costs more than it saves. dense_last_steps runs the final N sampling steps at full attention; 0 matches lightx2v exactly, 1 costs a little speed and can recover fine detail, since the last step's error is the one you actually see. protect_audio (on by default) always attends the [text | cond | audio] prefix - audio is ~1% of the packed sequence, so plain top-k regularly drops all of it and your soundtrack degrades while the video still looks fine. enabled toggles the whole thing for a like-for-like speed baseline without rewiring.

What people actually hit

The speech-artefact question comes up a lot. The author's finding: sparsity did not turn out to drive the speech artefacts on H3 - step count did, so use 6 steps rather than lowering sparsity to "fix" speech. And if a LoRA can't take 0.7+ sparsity, this node has nothing to offer it - don't force it.

Install

Same pack as the AdaLN fix, same install: search "PlagueKind-Nodes" in ComfyUI Manager, or

cd ComfyUI/custom_nodes
git clone https://github.com/PlagueKind/ComfyUI-PlagueKind-Nodes.git

Restart after. The Triton kernel needs Triton, which ships with standard torch CUDA installs - no separate dependency to fetch. If Triton is missing or something goes wrong on your GPU, the node catches it and passes the model through unchanged, so your run degrades to plain dense attention instead of dying. Check the console for [H3Utils] SLA: patch installed but never invoked - that warning means the patch is in but the attention path changed under you, and nothing is being sparsified.

CategoryPlagueKind/model_patches/minimax

Inputs (15)

Name	Type	Default	Description
model	MODEL		MODEL,
sparsity_ratio	FLOAT	0.800–0.95	Fraction of key blocks skipped. 0.85 is the shipped default and what the SLA turbo LoRA was distilled against; 0.90 is the value validated here and is ~15% faster. Sparsity did NOT turn out to drive the speech artefacts on H3 -- step count did, so use 6 steps rather than lowering this. Break-even is about 0.60 -- below that this kernel is SLOWER than dense attention,0.0 disables sparsity without removing the node.
block_size	COMBO	32	How many sequence tokens share one key selection. Unrelated to the model's 128-wide heads. This matters far more for audio than video: H3 packs audio at 80 rows per second, so a 128-row block forces 1.6 s of audio down one attention pattern, while the same 128 rows are only 3% of a video frame. Speech came out robotic at 128 and clean at 64, for about 2% more time -- halving the block doubles the block count, so the attention work is identical and only the routing gets finer. Use 128 only if you generate without meaningful audio. Coming down to 32 has increased the quality even further for marginal slowdown
min_seq_lenopt	INT	122280–1000000	Sequences shorter than this stay dense. Guards two things: the short text-refiner attention, which must never be sparsified, and low-resolution or short clips, where block selection would cost more than it saves. Lower it only if you know your sequence is long enough to benefit.
dense_last_stepsopt	INT	00–8	Run the last N sampling steps at full attention. 0 matches the turbo LoRA's original distillation setup exactly. 1 costs a little speed and can recover fine detail, since the final step's error is the one you actually see. Stacks with dense_steps below rather than replacing it.
protect_audioopt	BOOLEAN	false	Always attend blocks overlapping actual language tokens, target audio, and audio-reference segments. Visual-reference blocks are controlled separately. Disable only for turbo-style uniform sparsity; testing found partial or unprotected audio unstable for very little speed gain.
enabledopt	BOOLEAN	true	Turn off to pass the model straight through, for a like-for-like speed baseline without rewiring.
dense_stepsopt	STRING	1	Explicit 0-based step indices to force dense, on top of dense_last_steps -- e.g. '0,1' or '0-2'. Early steps set global composition and prompt adherence, so keeping just those exact and sparsifying the rest can fix prompt-following regressions without paying for full attention on every step. Blank = none.
dense_backendopt	COMBO	comfy_kitchen	Attention kernel used on every dense fall-through (short sequences, dense_last_steps, dense_steps). 'comfy_kitchen' (default, Comfy Kitchen int8) is fast enough on the handful of dense steps this node runs to be worth its precision tradeoff there. 'pytorch' pins the plain reference kernel instead if you want zero quantization anywhere in the dense path, at real cost to dense-step speed -- dense steps are the ones this node already decided must be exact, so that's the safest choice if you're chasing maximum quality over speed there. The 'sage:*' modes match an existing sage-attention kernel dropdown one-for-one -- 'sage:auto' lets the sageattention package pick; the rest pin a specific kernel + pv_accum_dtype: fp16_cuda uses fp32 accum (safest), fp8_cuda uses fp32+fp32 (safest of the fp8 pair), fp8_cuda++ uses fp32+fp16 (faster, more overflow-prone -- try this first if fp8_cuda works but you want more speed). Needs the sageattention package installed; falls back to whatever's already active with a logged warning if it isn't, or if a specific kernel is missing from your installed version. 'auto' restores the old behaviour of using whatever backend is already active globally. The sparse path is unaffected by this setting either way -- it never goes through backend selection at all.
disable_fp16_accumopt	BOOLEAN	true	Force off the fp16/bf16 reduced-precision matmul reduction path for this model's sampling run, regardless of the global --fast fp16_accumulation flag. Measured to cost quality on H3 with no throughput gain. Turn off only to A/B against the global flag.
stabilize_motionopt	BOOLEAN	false	Bias each layer's block selection toward what it picked last step, so a near-tie between two blocks doesn't flip for no reason and show up as a faint double-exposure on fast motion. Only target-video query rows are stabilized; text and audio choices remain step-local. It is a fix for that one specific symptom, not a general quality dial. - Uses more Vram
reference_protectionopt	COMBO	Off	Protect Image/Video Reference. Heavy Enforcement guarantees every Qwen vision-token, conditioning/image-reference, and video-reference block, matching the broad legacy prefix protection -- this is a reinforcement of those blocks, not a protection tuned for reference quality, and is most likely unusable with max ref size mode. Light uses fixed 0.85 reference sparsity and guarantees the best-scoring 15% of each visual-reference range. Off adds no special quota; references still participate in ordinary top-k. Default Off preserves the precise audio patch's fastest behaviour.
tail_correctionopt	BOOLEAN	false	Instead of a hard zero for every key block topk left out, fold in one pooled term standing in for all of them, so nothing leaves the softmax -- same idea as a widely-used block-sparse kernel's tail handling. Scored from the same pooled centroids selection already computes, so the only added cost is one more mean-pool of V plus a small reduction per call, not a second attention pass. Should help most at high sparsity, where the discarded tail is largest. Off by default: new, and its effect on H3 output quality specifically hasn't been validated the way sparsity_ratio's defaults have -- test before trusting it in a real render.
use_int8_qkopt	BOOLEAN	true	Quantize Q and K to int8 (per-token, dynamic scale) before the QK dot product on the selected topk blocks -- PV stays full precision (the mirror use_int8_pv toggle was removed from this node: its hidden widget could still be converted to an input socket and connected, which broke the node -- PV quantization is no longer exposed here). This is SageAttention's qk_int8_pv_fp16 split, not full int8 attention, and it's a different lever from dense_backend above: that setting only affects dense fall-through steps, this affects the sparse compute itself, on every sparse step. Ignored entirely when engine is comfy_kitchen, which quantizes internally regardless. UNTESTED ON HARDWARE: the quantize/dequantize math checks out against exact fp32 scores in isolation, but real speed, launch stability, and output quality on H3 have not been measured on a GPU. Try it against a known-good render before trusting it, and expect to possibly hit a launch failure on some GPU/Triton combinations before it's been shaken out.
engineopt	COMBO	comfy_kitchen	Which attention implementation runs the sparse path. triton (default) is this node pack's own kernel -- every other widget above applies to it fully. comfy_kitchen instead calls comfy_kitchen's real compiled sol_attn kernel (Comfy-Org/ComfyUI PR #16072, needs comfy-kitchen>=0.2.32 installed): genuine CUDA int8 compute and a built-in pooled tail term, but it can only express ONE contiguous protected range, has no reference-quota tier, and has no cross-step stabilize_motion -- reference_protection, multi-span protect_ranges, and stabilize_motion are silently disabled (one-time log warning each) rather than approximated, and tail_correction/use_int8_qk above are ignored since the real kernel quantizes internally regardless, but now honours tail_correction's on/off setting rather than always applying it. Both fall back to dense the same as any other kernel failure if their kernel is unavailable or throws.

Outputs (1)

Name	Type	Description
MODEL	MODEL	—