MinusZone - Flux1QuantizeUNETLoader

Every Quantize loader for Flux you've seen is probably GGUF. This one isn't, and that's the whole point. MinusZone - Flux1QuantizeUNETLoader loads a 4-bit AWQ/Marlin quantized Flux transformer straight into your sampler, using NVIDIA's Marlin kernels instead of llama.cpp's GGUF path. When Flux dropped in August 2024 and the 12B dev transformer priced itself out of consumer VRAM, the community split in two: most people went GGUF, and a smaller crowd wanted GPU-native int4 that doesn't pay GGUF's dequantization tax. This pack is that crowd's answer - one node, one model file, an approach you'll recognize if you've ever run a Marlin quantized LLM.

The name is doing you a small disservice. It says "UNET," but Flux doesn't have a UNet - it's a rectified-flow transformer. What the node really does is load a quantized Flux dev transformer from your unet folder. Under the hood it's a careful bit of surgery: it builds an empty Flux model config in bfloat16, walks the safetensors state dict looking for *.qweight tensors, swaps each matching linear layer for a WQLinear_Marlin 4-bit layer (group size 128), loads the rest with strict=False, and wraps the result in a ModelPatcher. The code hard-asserts 4-bit only, so don't go hunting for a Q5 of this.

What you actually set

The node has exactly one input, which is refreshing:

• unet_name - a dropdown of every .safetensors in ComfyUI/models/unet/. Pick the quantized file and go.

It returns a single model (MODEL) output, which wires straight into a KSampler the same way any loaded checkpoint's model does. That's the whole interface. Everything else about how it runs - the 4-bit math, the group size, the bfloat16 scaffolding - is decided for you.

Installing it, the real way

Manager can find it as ComfyUI-Flux1Quantize-MZ, or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/MinusZoneAI/ComfyUI-Flux1Quantize-MZ

Then restart ComfyUI and don't skip the part the README warns about. This pack has no requirements.txt, so Manager won't install its one real dependency for you - you have to:

pip install git+https://github.com/IST-DASLab/marlin

That's the Marlin kernel package, and without it the node just asserts "Marlin kernels are not installed." The model file itself is the other half: grab flux1-dev-unet-marlin-int4.safetensors from MinusZoneAI/flux1-quantize on HuggingFace (or the wailovet/flux1-quantize Modelscope mirror) and drop it in ComfyUI/models/unet/. Note the README's hardware line: sm_80 and newer only, i.e. RTX 30-series and up. Marlin is a CUDA kernel - no AMD, no Intel, no Mac, unlike GGUF.

You still need the rest of a normal Flux workflow. This file is only the transformer, so pair it with the T5 and CLIP text encoders, the VAE, and Flux dev's usual guidance (try around 3.5 in the sampler).

Where people get burned

• Empty dropdown - the file isn't in models/unet (not models/checkpoints), or you haven't restarted after dropping it there.
• "Marlin kernels are not installed" - the pip install went to the wrong Python. Install it into the exact environment ComfyUI runs in.
• Compile failures on that pip install - marlin builds a CUDA extension; you need a working CUDA toolchain and a matching driver. It's the fiddliest part of this whole pack, and it's why most people gave up and used GGUF.
• Wiring it into an SDXL sampler - this is a 12B Flux transformer, not a UNet. It needs a Flux graph, period.

Honest take: if you already run Flux fine on fp8 or GGUF Q8 (which the community calls "99% identical to fp16 at half the VRAM"), this won't change your life - it's an int4 path for people who want Marlin's speed on an RTX 30/40 card and don't mind one afternoon of setup. It's also an archaeology piece: one commit, August 2024, and the ecosystem moved on. But if you're curious how the non-GGUF half of Flux quantization worked, this is the cleanest example of it.