ComfyUI Node
Patch Sol-Attn (SGLang)
Worker-native replacement for Patch Sol-Attn.
Patch Sol-Attn (SGLang)
- model
- MODEL
◄tau1.30►
◄start_percent0.20►
◄end_percent0.90►
◄min_tokens4096►
◄int8_qktrue►
◄sink_conditioningexact_kv_and_rows►
◄mortonfalse►
◄morton_curve2d_frame►
◄int8_pvtrue►
◄verbosefalse►
◄use_tmafalse►
◄dense_blocks►
◄tau_profile—►
CategorySGLang/MiniMax H3
Inputs (14)
| Name | Type | Default | Description |
|---|---|---|---|
| model | MODEL | — | |
| tau | FLOAT | 1.300–4 | Threshold beta. Higher is sparser: 1.0 ~ 16% of blocks kept exact, 1.5 ~ 7%, 2.0 ~ 2.7%. |
| start_percent | FLOAT | 0.200–1 | Run dense before this point. The paper uses 0.2. |
| end_percent | FLOAT | 0.900–1 | — |
| min_tokens | INT | 40960–1048576 | Sequences shorter than this stay dense. |
| int8_qk | BOOLEAN | true | INT8 QK in the exact branch (Sage-style: smoothed K, per-token scales). Measured free in quality; helps at tau<=1.5, a net loss at tau>=2.0 where the quantize pass outweighs the shrinking exact branch. |
| sink_conditioning | COMBO | exact_kv_and_rows | MiniMax-H3 only. exact_kv: every query sees the packed text/audio/reference rows exactly (~3% cost). exact_kv_and_rows: also runs those query rows dense, making the generated audio stream exact (~20% cost). No effect on other models. |
| morton | BOOLEAN | false | Reorder video tokens into Morton (Z-order) so each 64-token block is a compact 3D neighbourhood instead of a 2-row strip, which makes routing far more accurate at a given density. Exactly neutral for dense attention. Wan and MiniMax-H3 only; logged and skipped elsewhere. |
| morton_curve | COMBO | 2d_frame | 3d interleaves t/h/w equally. 2d_frame Z-orders within each frame and leaves frame order alone -- use it when the temporal axis is not uniformly spaced (MiniMax-H3's frame spacing is non-uniform; try this if 3d degrades at some frame counts). |
| int8_pv | BOOLEAN | true | Also run the exact branch's P@V in INT8, with a per-row P scale and per-channel V scale. PV and QK cost the same, so this is the other half of the int8 win. Only applies when int8_qk is on. |
| verbose | BOOLEAN | false | — |
| use_tma | BOOLEAN | false | Use the TMA descriptor kernels instead of the pointer ones. Descriptors address strided inputs directly, so this no longer copies q/k/v and peak VRAM matches the pointer path. Off by default because it has not measured faster on any tested GPU. Requires SM90+ and Triton 3.3+; ignored otherwise. 'verbose' logs the path used. |
| dense_blocks | STRING | Transformer blocks to keep dense, e.g. '0-2,-1' for the first three and the last. Negative indices count from the end. The first and last blocks are the most approximation-sensitive: their error reaches the output with no later block to absorb it. Empty means sparsify all. | |
| tau_profileopt | STRING | Per-block tau, overriding the base value. 'blocks=tau' entries separated by ';' or newlines, so a multiline text node works: '0-30=2.0' then '39-42=0.9'. '#' starts a comment. Block sensitivity varies several-fold across depth, so one tau either over-serves the insensitive blocks or under-serves the fragile ones — use the Block Probe to find them. Leave unconnected for a single tau everywhere. |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| MODEL | MODEL | — |