FLUXATTENHUT
The tuning knobs behind FLUXNATION's spike attention
- model
- model
FLUXATTENHUT is the node where FLUXNATION's "spike attention" actually gets tuned. FLUXNATION is a fused FP8 CUDA kernel that replaces FLUX.1's SingleStreamBlock forward pass in ComfyUI, with a block-sparse "spike" attention pass and step caching bolted on top to cut generation time on RTX 40-series cards. The kernel patches itself into ComfyUI at startup and does its work whether or not any node is in your graph - so these nodes exist to give you live control over it, and FLUXATTENHUT is the dial panel. You wire a MODEL into it and it hands the same MODEL back out; all it does is push three floats into the kernel's controller.
The three inputs are the whole story:
switch_pct(default 0.50) - the fraction of the denoising run that passes before spike attention kicks in. 0.50 on a 20-step run means it activates at step 10. The first half of sampling builds image structure, and the author's argument is that by then the attention is expensive and mostly redundant. Lower it to let the sparsity take over earlier (more speed, more risk), raise it to keep dense attention longer (safer, slower).cap_pct(default 0.45) - how much attention you keep once spike is active. Only the top 45% of attention blocks, ranked by a dot-product similarity score, actually get computed. This is the speed/quality lever. Drop it toward 0.1 if you're chasing every millisecond, push it up if detail softens.tau(default 0.05) - the block-scoring threshold. Blocks whose score falls belowtautimes the row max are skipped. Lower tau keeps more blocks; it's the fine-grained companion tocap_pct. The tooltip says it plainly: lower = more blocks kept.
That's it. Feed it a model, set the numbers, and the same model comes out the other side. It's a configuration node, not a processing node.
How the mechanism works matters here, because it tells you which knob to turn when something looks off. After switch_pct of the diffusion steps, each attention layer starts scoring its 32-token blocks with a real q_rep · k_rep^T dot product across heads, keeps the top cap_pct of them, and skips the rest - the README claims roughly 90% attention FLOPs saved on spike steps. The last two model evaluations always run dense, so the tail of the denoise still gets full attention. The Triton kernel does this block-sparse pass; if Triton isn't available it falls back to a pure-PyTorch spike path instead.
Where people get burned: the node's defaults and the startup environment variables can disagree. The launch script run_nvidia_gpu_spike.bat sets FLUX_SPIKE_SWITCH=0.50, FLUX_SPIKE_CAP=0.45, FLUX_SPIKE_TAU=0.05 - which match these node defaults - but the pack's own SPIKE_CONFIG reads 0.35/0.35/0.05 from the environment if you launch plain. The env vars set the baseline at startup; whatever you type into this node overrides it live for that session, no restart. If you set values here and nothing changes, check which way you launched - the console line [FLUXATTENHUT] switch=50% cap=45% tau=0.05 tells you what the node actually pushed.
Install is the pack install, which is the fiddly part of FLUXNATION: it's not a pip install. The README's route is to clone the source, copy the node folder into ComfyUI/custom_nodes/, then compile the CUDA extension:
git clone https://github.com/ULT7RA/cuda-kernels.git
# copy FLUXNATION/custom_nodes/FLUXNATION/ -> ComfyUI/custom_nodes/FLUXNATION/
cd FLUXNATION
build_ext.bat
That build needs Visual Studio 2022, CUDA Toolkit 12.x, and Python 3.10 - the setup script hardcodes -arch=sm_89, i.e. RTX 4090-class Ada, and it expects CUTLASS headers in a cutlass/include folder that isn't actually shipped in the repo, so you'll likely need to drop NVIDIA/cutlass in there before it compiles. There's no requirements.txt; the node leans on the torch and Triton already in your ComfyUI environment. On Linux the README's fallback is python setup_ext.py install from the node directory. ComfyUI Manager can trigger the build via the pack's install.py, but the pack is too new and obscure (zero traffic on comfy.icu) to count on it being listed.
This is honest-to-goodness "I hand-tuned a kernel for my 4090" territory, and the spike attention is genuinely clever - but it's also an unproven, single-author pack with no real community validation yet, and it needs a working CUDA toolchain just to try. If the build fails, nothing blows up: the node prints that the extension is disabled and your model just passes through at stock speed. Treat FLUXATTENHUT as a lab instrument, not a default you put in every workflow.
Inputs (4)
| Name | Type | Default | Description |
|---|---|---|---|
| model | MODEL | — | |
| switch_pct | FLOAT | 0.500.1–0.9 | Fraction of steps before spike activates (0.50 = step 10/20) |
| cap_pct | FLOAT | 0.450.1–0.9 | Fraction of attention blocks to keep (0.45 = 45%) |
| tau | FLOAT | 0.050.01–0.5 | Block scoring threshold — lower = more blocks kept |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| model | MODEL | — |