LC Model Quantize 🗜️

Why this exists

Quantized downloads are plentiful - Comfy Org hosts ConvRot INT8 conversions of the current roster, city96 and bullerwins keep the GGUF ladder stocked, and there's usually an fp8 within a day of any release. What nobody hosts is your model: the merge you just built, the checkpoint with two LoRAs baked in, the finetune you're too polite to upload at 25 GB.

This node makes the small version locally, from the file you already have or straight off the MODEL wire.

How it works

You pick a source and it streams. From a file, weights are read one tensor at a time, quantized on the GPU and written straight to disk - so even a model bigger than your RAM works (the author's note: Flux 2 dev on 32 GB). From the MODEL wire, your merges and LoRA patches are baked in as they go, which means the bf16 model has to fit in system RAM first - roughly 2 GB per billion parameters, fine up to 12-14B on 64 GB.

The sensitive layers stay full precision, the way released quants do: embedders, norms, modulation, the final layer, anything that isn't a 2-D weight or is too small to survive. protect_edges extends that to the first and last transformer block for a little more size and a little more fidelity. The two 4-bit formats also round activations, which breaks layers fed by the timestep and the text side - those stay full precision there too (on Anima, without that, you get pure noise).

The formats

Native ComfyUI files, loaded with the stock Load Diffusion Model: fp8 (scaled), mxfp8, nvfp4, int8 (convrot), int4 (convrot w4a4).

GGUF files, loaded with ComfyUI-GGUF's Unet Loader (GGUF): Q8_0, Q6_K, Q5_K_M/S, Q4_K_M/S, Q3_K_M/S, Q2_K.

Which one: int8 (convrot) and Q8_0 come out closest to bf16, and int8 is the one that runs on a 30-series card with no fp8 hardware at all. fp8 and Q5_K_M are next. nvfp4 and mxfp8 are 50-series territory. Below Q4 you're solving a fit problem, not a quality one. On a 12B model the author measured everything from fp8 down to Q2_K still rendering - the layout shifts, nothing breaks - but on small models it falls apart fast: Anima at 2B is rough at Q3 and noise at Q2_K, and the report warns you below 6B.

Inputs and outputs

source is either (use the model input) - then wire model - or a file from diffusion_models / checkpoints (an all-in-one checkpoint gives up just its diffusion model). format is the list above. protect_edges as described. filename is where it lands under models/diffusion_models; empty means the source name plus the format, subfolders like Krea2/Krealism work, and an existing file is never overwritten.

The single output is report - what it wrote, the size, the bf16 original's size, and the drift. Drift is the average distance of the quantized weights from the originals relative to their size: 0% is identical, lower is closer. Read it as a smoke check, not a quality score. fp8 at 2.2% and Q8_0 at 0.6% both look the same as bf16 in a render.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/lonecatone23/ComfyUI_LC_ModelBuilder_nodes

Restart. No dependencies for the native formats - the GGUF quantizers are ported from llama.cpp and live in the pack itself. But to load a GGUF you produce, you need ComfyUI-GGUF installed, same as any other GGUF.

Where people get burned

Quantize from bf16 or fp16. An fp8 or GGUF source gets decoded first, but its rounding stacks with the new one and you're measuring noise. The report warns when it detects that.

It checks free disk space before it starts and stops if there isn't enough, which is kinder than the usual halfway-through-a-write failure. And the last one is a judgement call rather than an error: don't quantize a model that already fits. If your card holds the bf16, the smaller copy buys you disk space and nothing else.