INT8 Lazy Torch Compile

The README says it plainly and it's true: torch.compile is often the difference between "INT8 works" and "INT8 is actually fast." Quantizing your weights gets you the VRAM savings; compiling is what turns the matmuls into fast fused kernels instead of a bunch of separate ops. The catch is that ordinary torch.compile nodes compile whatever graph exists at the moment you call them - which, if that's before your INT8 patches are installed, means you've compiled the wrong thing. This node exists specifically to compile after ComfyUI's own object patches, INT8 included, are already in place.

How it works

"Lazy" means it doesn't compile the moment the node runs - it waits until the first actual sampling call, by which point Enable INT8 on MODEL (or the INT8 loader) has already swapped in its quantized layers. The recommended placement, straight from the README, is Enable INT8 on MODEL → INT8 Lazy Torch Compile → sampler.

This isn't unique to INT8 - torch.compile in ComfyUI generally is a well-worn speed lever people chase across GGUF and Flux workflows too, with real reported gains once it's dialed in. It's also famous for two annoyances that apply here as much as anywhere: your first generation after adding compilation eats a slow compile pass before anything runs, and dynamic shape changes can trigger expensive recompilation. This node's specific answer to the second problem is compile_transformer_blocks_only - instead of compiling the whole model graph, it targets the repeated transformer block list most diffusion architectures are built from, which is both faster to compile and captures most of the actual runtime, since a diffusion model spends most of its time looping the same block. It also applies "Comfy-style guard filtering" so it works with ComfyUI's dynamic patching instead of fighting it, and lets you raise the Dynamo cache limit for workflows juggling a lot of compiled modules.

The inputs and outputs that matter

• model - your INT8-converted model, coming out of Enable INT8 on MODEL or the INT8 loader.
• backend - inductor (default, the standard PyTorch compiler) or cudagraphs. Stick with inductor unless you have a specific reason to try the other.
• mode - default first; max-autotune spends more compile time hunting for faster kernels, worth trying once the base setup is confirmed working.
• compile_transformer_blocks_only (default true) - the README's recommended default. Only flip it off for an architecture that specifically needs whole-model compilation.
• dynamic (default true) - lets input shapes vary without forcing a fresh recompile every time your resolution or batch size changes.
• dynamo_cache_size_limit (default 640) - raise it if you hit a Dynamo cache limit error with many compiled modules in play.

Output is a single MODEL, wired straight into your sampler.

How to install it

• ComfyUI Manager - search "ComfyUI-INT8-Fast-Fork", install, restart.
• Manual - cd ComfyUI/custom_nodes && git clone https://github.com/SparknightLLC/ComfyUI-INT8-Fast-Fork, then restart.

No model downloads needed for this node specifically. You do need a PyTorch build with a working torch.compile for your platform - this is one of the pack's heavier dependencies, and on Windows the compile backends can be fussier to get running than the quantization itself.

Common issues & troubleshooting

First generation after adding this node is much slower. That's the compile happening, not a problem. It's a lazy, first-call compile by design - judge speed from the second generation onward, not the first.

A compile experiment failed, and now things seem broken even after you change settings back. The README's own advice: restart ComfyUI. A failed or partial compile can leave stale compiled state hanging around that a settings change alone won't clear. Don't draw conclusions about a config until you've tested it after a clean restart.

Placement - put it after Enable INT8 on MODEL, not before. Compiling before your INT8 conversion compiles the pre-quantized graph, so the fast INT8 kernels never actually get compiled in. The node is built to be lazy specifically so this ordering works, but you still need to wire it after the conversion node, not before.

Hitting a Dynamo cache limit warning or error with a complex workflow. Raise dynamo_cache_size_limit above the 640 default - workflows with several compiled modules (multiple models, ControlNets, etc.) can exceed it.

Should I leave compile_transformer_blocks_only on? Yes, unless your specific architecture is known to need whole-model compilation. It's the README's stated default for a reason - narrower compilation is faster to set up and covers the part of the model that actually dominates runtime.