H3 Sparse Attention

MiniMax H3 is a 33B omni-modal model that generates picture and sound in one pass, and it's lovely right up until you ask for 124 frames at 1344x768 - roughly 38,000 tokens per attention call. Attention is where long clips go to die, in seconds-per-step and in VRAM alike. H3 Sparse Attention is the knob for that: most of H3's attention layers compute a fraction of the query-key pairs they normally would, while the handful that visibly matter run exactly as they did.

It ships in WAS Node Suite - 518 nodes, MIT, and the pack installs nothing. One licence note first: the H3 weights are under a community licence that excludes the US, EU, UK and Korea. If you're in one of those, none of this is licensed for you to run locally, and no node fixes that.

What it actually does

H3's diffusion transformer is 50 blocks, and the node patches them one at a time. Each block in the plan either runs dense on the model's own attention backend or goes through the Sol-Attn kernel - sol_attn_chunked from comfy_kitchen, which ships with recent ComfyUI. Inside the kernel, work happens in 64-row blocks and sparse layers compute in INT8. That quantisation is where the softness comes from.

Two things follow. This is H3-only: there's a core ComfyUI Model Sparse Attention node for everything else, and feeding a Wan or Flux model to this one raises a ValueError naming the type it got. And it degrades quietly rather than failing. If the token count is under min_tokens, the sigma is outside your start/end window, the model isn't bf16, you're not on CUDA, or the kernel can't run on your card, that layer just runs dense and logs a warning. The plan output is how you find out which.

The inputs that matter

model is the H3 model after any LoRA - the pack's example workflow puts a 4-step distilled LoRA in front. Wire it into a BasicGuider.

method is a dynamic combo, so picking it changes what you get next. sol-attn (default, any H3 weights) gives you tau: 1.0 keeps roughly 16% of key blocks exact, 1.5 drops to about 7%, 2.0 goes to about 3% and is the fastest and softest. sla and vsa instead give you keep_percent, default 10 - the share FastH3 was trained at. Use sla for SLA-distilled LoRAs, vsa for FastH3 weights.

layers picks what stays dense: balanced (default) holds layers 39-42 and 49, careful holds 0-1, 37-43 and 49 for about 2x instead of 2.3x, fastest makes everything sparse for a claimed 2.7x. Those are the author's numbers on a 4090 with sol-attn at tau 1, not universal. manual opens a layer_plan box, where 39-42, 49 runs those dense, 0-3: 0.5 gives layers their own setting, 20: dense does one, and later entries win.

start_percent (0.2) and end_percent (1.0) bound the window in the noise schedule. The default keeps the first fifth of the steps dense - a quality decision, not an accident.

Three advanced ones sit below that: sink_conditioning (default exact_kv_and_rows reads the prompt, reference and audio rows exactly and runs the audio rows dense - turn it off and you're routing your prompt like video), extra_tokens (256, the top-scoring tokens each query block reads beyond its chosen blocks; 0 is fastest), and min_tokens (12288, below which calls run dense).

Install

Easiest via ComfyUI Manager: search WAS Node Suite v3. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/WASasquatch/was-node-suite-comfyui.git

Restart, done. The pack needs ComfyUI 0.14.0+ and Python 3.10+, and installs nothing - no pip, no git URLs, no model downloads, and outbound requests are off by default.

The one hard requirement is the GPU. Sol-Attn is NVIDIA-only and wants sm_80, i.e. RTX 30-series or newer. On anything else the node loads, runs, and does nothing - every layer dense.

Where people get burned

Check the plan string into a Preview Any or Show Text node before you trust a render. It reports exactly what happened, including Every layer runs dense here: <reason>. Two reasons come up constantly: the model was loaded in something other than bf16 (weight_dtype: default in the UNET loader, not fp8), or the kernel isn't available on the card.

Short clips won't speed up at all - under min_tokens there's nothing to gain, and 1344x768 at 124 frames is only about 38k tokens. And if a call runs out of memory at a given sequence length, the node blacklists that token count and runs it dense for the rest of the run, which looks like the speedup evaporating halfway through a graph.

The pack ships an example graph at docs/workflows/minimax-h3-sparse-attention.json that samples the same seed twice - one dense branch, one sparse - and compares the two clips with WAS Video Compare. Run that on your own footage first. Whether 2.3x is worth the softening is a judgement about your shot, not a setting someone else can pick for you.