INT8 Kernel Config

Be honest with yourself about whether you need this one. It's a fixed-parameter tuner for the Triton runtime backend's matmul kernels - block sizes, warp counts, pipeline stages, the kind of thing you'd normally leave to Triton's own autotuner. If you're running the default torch_int_mm backend elsewhere in your graph, this node has nothing to tune and does nothing useful. It only matters once you've switched to triton and are chasing the last bit of throughput on a specific GPU and shape.

How it works

Instead of letting Triton pick kernel parameters at runtime, this node applies fixed values you supply: block_m, block_n, block_k (the matmul tile dimensions), group_size_m (thread-block grouping for cache locality), num_warps, and num_stages (software pipelining depth). The defaults (128/128/64, group size 8, 4 warps, 4 stages) are reasonable starting points, not magic numbers tuned to your card.

If you don't know what values to pick, that's what run_microbench is for. Turn it on and the node benchmarks candidate configs against synthetic matmul shapes you control via bench_m, bench_k, bench_n, bench_warmup, and bench_iterations - then prints the environment-variable values it found fastest. The intended workflow is: run the microbench once, note the printed values, then set this node's fixed parameters (or the equivalent env vars) to match and leave run_microbench off from then on. Running a full benchmark sweep on every model load would be its own performance tax.

The inputs and outputs that matter

• model - pass-through; the node applies the kernel config as a side effect and returns the model.
• run_microbench - off by default. Turn on once to find good values, then turn back off.
• block_m / block_n / block_k, num_warps, num_stages - the actual kernel parameters, only worth touching after you've microbenched or have a known-good config for your GPU.

Output is a single MODEL, passed through with the runtime config applied.

How to install it

• ComfyUI Manager - search "ComfyUI-INT8-Fast-Fork", install, restart.
• Manual - cd ComfyUI/custom_nodes && git clone https://github.com/SparknightLLC/ComfyUI-INT8-Fast-Fork, then restart ComfyUI.

The one thing worth flagging: this node's settings only apply if you're actually using the Triton runtime backend elsewhere (set on Enable INT8 on MODEL or Load Diffusion Model INT8 (W8A8)). That means you need Triton installed and working - on Windows specifically, the README calls out triton-windows matched to your PyTorch/CUDA build (their tested combo: PyTorch 2.8.0+cu126 pulling in Triton 3.4.0 via triton-windows 3.4.0.post21). Get a mismatched version and this node silently has nothing to do.

Common issues & troubleshooting

Nothing seems to change no matter what I set. First check whether runtime_backend is actually set to triton on your model-loading node. On torch_int_mm (the default backend), this node's parameters are inert - there's no Triton kernel for them to configure.

Errors on Windows specifically. Almost always a Triton/PyTorch/CUDA version mismatch. Match your triton-windows package to your installed PyTorch build; don't assume the latest version of each is compatible with the other.

Should I leave run_microbench on all the time? No - it's a one-shot diagnostic. Run it, read the printed recommended values from the console, apply them (or hardcode them into this node), then switch it back off. Leaving it on adds real benchmarking overhead to every model load.

Honestly, do I need this node at all? Probably not on your first pass. This is the one node in the pack worth skipping until you've got INT8 working end to end on the default backend and you're specifically hunting for more speed on a known GPU. Most of the pack's value - the VRAM savings, the speed jump on older cards - comes from Enable INT8 on MODEL or Load Diffusion Model INT8 (W8A8) alone.