ComfyUI-INT-Crush

ComfyUI loader for INT-Crush quantized models (INT4 + INT8). Supports models quantized with INT-Crush Converter

⚠️ WARNING: This code has not been thoroughly tested.

Developed with AI assistance.

Installation

Clone this repository into your ComfyUI custom_nodes/ directory:

cd ComfyUI/custom_nodes
git clone https://github.com/ThunderFun/ComfyUI-INT-Crush.git

Nodes

INT4 UNet Loader (INT-Crush) — SimpleINT4UNetLoader

| Input | Description | |-------|-------------| | unet_name | Quantized .safetensors from models/diffusion_models/ | | rot_size | 0/16/64/256/1024/4096 (auto-detected from metadata) | | kernel_backend | auto / triton / cuda / pytorch (default auto) |

INT8 UNet Loader (INT-Crush) — SimpleINT8UNetLoader

| Input | Description | |-------|-------------| | unet_name | Quantized .safetensors from models/diffusion_models/ | | rot_size | 0/16/64/256/1024/4096 (auto-detected from metadata) | | kernel_backend | auto / triton / cuda / pytorch (default auto) |

OrbitQuant UNet Loader (INT-Crush) — OrbitQuantLoader

| Input | Description | |-------|-------------| | unet_name | OrbitQuant .safetensors from models/diffusion_models/ |

LoRA Loader (INT-Crush) — IntCrushLoRALoader

| Input | Description | |-------|-------------| | model | MODEL output from an INT4/INT8 loader | | lora_name | LoRA file from models/loras/ | | strength | LoRA strength (default 1.0) |

LoRA Unloader (INT-Crush) — IntCrushLoRAUnloader

| Input | Description | |-------|-------------| | model | MODEL with INT-Crush LoRA buffers to remove |

Workflow

# 1. Quantize (from int_crush_converter/)
python -m converter.cli -i model.safetensors -o ./out --rot-size 256 --int-bits 4

# 2. Copy to ComfyUI
cp ./out/model.safetensors /path/to/ComfyUI/models/diffusion_models/

# 3. Load with INT4/INT8 UNet Loader node (auto-detects rot_size from metadata)

Inference

The inference path is selected based on the kernel_backend toggle, kernel availability, and batch size.

INT4

Three backends, selected by kernel_backend:

CUDA backend (comfy-kitchen) — preferred when available:

• Unpack INT4→INT8 via CUDA kernel → INT8 GEMM (cuBLASLt / CUTLASS) with fused dequant
• For rot_size=256 layers without PermuQuant/SmoothQuant: fused ConvRot path — Hadamard rotation + activation quantization + INT8 GEMM in a single CUDA kernel
• For other rot_size values or layers with PermuQuant: separate rotation (PyTorch) → INT8 GEMM

Triton backend — fallback when CUDA unavailable:

• W4A8 (fastest, rot_size > 0): Triton INT4→INT8 unpack → dynamic-quantize activations → fused INT8 GEMM + dequant
• W4A16 (rot_size == 0): Triton INT4→float16 unpack → cuBLAS GEMM

PyTorch backend — always available (slowest):

• Full dequant to float → F.linear

INT8

CUDA backend (comfy-kitchen) — preferred when available:

• Direct INT8 GEMM (cuBLASLt / CUTLASS) with fused dequant — activation quantized inside the kernel
• For rot_size=256 layers without PermuQuant/SmoothQuant: fused ConvRot path (rotation + quantize + GEMM in one kernel)
• For other rot_size values or layers with SmoothQuant/PermuQuant: separate rotation → INT8 GEMM

Triton backend — fallback when CUDA unavailable:

• Dynamic per-token INT8 quantization → fused INT8 GEMM + dequant
• CUDA graph capture available for repeated shapes (eliminates launch overhead)

PyTorch backend — always available (slowest):

• Full dequant to float → F.linear

OrbitQuant

OrbitQuant uses a different quantization scheme (Lloyd-Max codebook + RPBH rotation) that is not compatible with the CUDA/Triton INT8 GEMM paths. OrbitQuant layers always use:

• Fused Triton codebook GEMM (small M) or dequant to float → cuBLAS (large M)
• RPBH rotation via Triton kernel or PyTorch fallback
• The kernel_backend toggle does not affect OrbitQuant layers

LoRA

IntCrushLoRALoader attaches LoRA as a residual buffer in unrotated activation space. This avoids corrupting the quantized weight's Hadamard rotation by applying the LoRA before any rotation or smoothing. The low-rank matrices (A, B) are stored on CPU and moved to GPU on first forward. Standard ComfyUI LoRA patching is not compatible with INT-Crush models — use the INT-Crush LoRA nodes instead.

Weight Format

INT4: <name>.weight (uint8, 2 per byte) + <name>.weight_scale (fp16, [out, num_groups]) + optional <name>.perm

INT8: <name>.weight (int8) + <name>.weight_scale (fp16, [out, 1]) + optional <name>.perm

Forward Pass

1. SmoothRot (if SmoothRot factors present) — divide activations by scale factors before rotation
2. Rotate activations (Hadamard or RPBH, if applicable)
3. Pad to weight in_features
4. Apply PermuQuant permutation (if present)
5. Quantize weights and activations, then GEMM (backend-dependent)
6. Apply SVD low-rank residual (if L1/L2 factors present)
7. Apply adapter residual (LoRA / LoKr / LoHa / OFT / BOFT)

When using the fused ConvRot CUDA path (step 2), steps 2 and 5 are combined into a single kernel call for rot_size=256 layers.

Notes

• ~4× compression (INT4), ~2× (INT8) vs fp16.
• Padded layers auto-detected and fixed.
• rot_size is auto-detected from safetensors metadata (int_crush.format_version / int_crush.rot_size).
• comfy-kitchen CUDA backend requires no Triton installation — works with just PyTorch + CUDA.
• RPBH and OrbitQuant layers still require Triton for their rotation kernels (or fall back to PyTorch).