XB-BOX - 🧊 INT8 UNet 加载器 (ROCm)

Quantization is how big models end up on small cards - the GGUF Q-ladder made Flux run on 12GB, and INT8 is the next rung down the same ladder. XB_UNetLoaderINTROCm is the XB_ToolBox INT8 loader: it loads an INT8-quantized diffusion model into ComfyUI, hooks the model's linear layers up to custom INT8 kernels, and (with the right switches) even quantizes on the fly so you can run a model without waiting for a pre-quantized file. The "ROCm" in the name is the giveaway that this was built for the AMD branch of the ecosystem, though it runs wherever the kernels can execute.

The honest framing up front: this is a loader for a specific quantization ecosystem. It does not load GGUF files - it loads INT8 .safetensors diffusion models produced by the pack's own INT8 save/quantize nodes (INT8ModelSaveROCm and friends), so the full workflow is "quantize once, then load with this node." And it's a piece of the pack's bigger story: 14B+ video and image transformers, quantized to INT8, driven by Triton kernels.

Inputs that matter

• unet_name - pick the INT8 model from models/diffusion_models.
• weight_dtype - compute dtype: default / fp16 / bf16 / fp32. This is the precision the math runs at, not the stored weights. bf16 is the safe modern default; fp32 if you're chasing numerical correctness on ROCm.
• model_type - the critical one: flux2, z-image, ideogram4, chroma, krea2, wan, ltx2, qwen, ernie, anima, hidream o1, or boogu. It tells the loader which layers to keep out of INT8 - each architecture has its own exclusion list (embeddings, modulation layers, the final projection) so the sensitive parts stay in higher precision. Pick wrong and quality degrades or the run breaks.
• on_the_fly_quantization - off by default (load pre-quantized weights). On, it quantizes the model as it loads, so you can feed it an unquantized model and still get INT8 memory usage.
• enable_convrot - on by default: rotation-based (Hadamard) transform for the convolution layers, a trick that makes INT8 convs behave closer to fp16.
• lora_mode - None / Stochastic / Dynamic, for baking LoRAs into the quantized model. Stochastic uses random-rounding deltas; Dynamic re-quantizes per layer at load. Pair with the pack's PreLoraLoaderROCm (PRE_LORA) for the "bake at load" workflow.

Output is a MODEL - wire it to your sampler like any other.

The mechanism, briefly

The loader attaches Int8TensorwiseOps custom operations to the model: matmuls route through Triton INT8 kernels that multiply in INT8 and dequantize back on the fly. Two real prerequisites: Triton (the source falls back to a hard error if it's missing) and, for the advanced Aimdo dynamic-loading path, comfy_aimdo. The INT8 kernels are the speed story too - the whole point is fewer bytes through the memory bus, which on bandwidth-starved cards can be as much about speed as about fitting.

Where people get burned

Three classic failure points: model_type mismatch (quality collapses or kernels hit an unexpected layer shape), trying to load a plain GGUF or fp16 file into this node expecting magic (it's for the INT8 ecosystem - pre-quantize first), and missing Triton on an ROCm install where the wheels don't cooperate. And the eternal quantizer's law from the GGUF world applies here too: Q8/INT8 is where loss is still basically invisible; this is not the node to reach for if you have the VRAM to run fp16 and just want quality.

Install: Manager → XB_ToolBox, or git clone https://github.com/wjluoxiao/XB_ToolBox.git into custom_nodes, restart. This node needs Triton (and optionally comfy_aimdo) installed in your Python env.