Targeted Quantization (GGUF)

Normally you quantize a model with a command line, then load the result. Targeted Quantization (GGUF) moves that whole step into your workflow - point it at a source checkpoint, tell it a size, and it writes you a GGUF file you can load right back in with Unet Loader (GGUF). It's the node version of the pack's tools/convert.py, with one genuinely clever trick: the TARGET_SIZE mode that does mixed quantization to land under a size cap instead of just slapping one quant level over everything.

Why mixed matters: a uniform Q4 model is small, but a uniform Q8 is safer. Targeted tries to have both. It starts every eligible 2-D Linear weight at INT8 (Q8_CR by default), protects the 1-D and architecture-sensitive tensors in FP32, then walks core matrices toward the model's center down to Q5_0 and then Q4_0 until the target size is hit - keeping the beginning and end of the network at high precision as long as possible. The idea is the parts of the network that handle early features stay accurate while the middle takes the compression. You get a smaller file than flat Q8 and usually better output than flat Q4.

The inputs that matter

• source_path - absolute path to a .safetensors, .ckpt, .pt, .bin, or .pth. It must exist on disk; this is a filesystem path, not a model dropdown.
• quantization - TARGET_SIZE, or a fixed level: F16, BF16, Q8_0, Q5_1, Q5_0, Q4_1, Q4_0, or Q8_CR.
• max_size_mb - required only for TARGET_SIZE. That's your budget. If the target is below what's physically achievable, it errors and tells you the minimum.
• target_size_q8_type - the baseline for TARGET_SIZE: Q8_CR (native INT8 ConvRot) or Q8_0 (standard GGUF Q8).
• quantization_device - auto (CUDA when available, CPU fallback per-matrix), cpu, or cuda.
• overwrite - whether to clobber an existing destination file.
• streamed - for safetensors sources, process one tensor at a time and stage on disk to keep RAM/VRAM down on big models.
• Optional lora_paths / lora_strengths - absolute paths to .safetensors or .gguf LoRAs to fuse in before quantization. Blank strengths default to 1.0.

Outputs are two strings: gguf_path (where the file was written) and quantization_info (a summary like Q8_CR: 6432.19 MiB written to /path). Destination is derived from the source if you leave destination_path empty.

Installing

Same pack, same drill - search ComfyUI-GGUF in the ComfyUI Manager or:

cd ComfyUI/custom_nodes
git clone https://github.com/molbal/ComfyUI-GGUF

Restart, pip install --upgrade gguf. The converter runs from the workflow, so you need the node to load, but no extra model downloads.

Where people get burned

• TARGET_SIZE without a size = instant error. The node raises if max_size_mb isn't greater than zero when you select it.
• The quality ladder is real, but it's not linear. Q8 is essentially indistinguishable from fp16, Q5 is the last tier where most people can't see a difference, and Q4 is where specific detail - text, logos - starts to go. This node's whole pitch is that you don't have to take Q4 everywhere.
• _K quants aren't available here for diffusion models, and Q4_0 is the floor - the converter won't go to Q3/Q2, and it'll tell you so.
• On NVIDIA 30-series, Q8_CR is the house recommendation - native INT8 ConvRot runs fast on cards with no fp8 acceleration, which is most of the reason this fork exists.
• LoRA fusion is applied before quantization; an active LoRA later in the workflow also keeps Q8_CR layers off the native INT8 fast path at inference time.