load_GGUF

If a 12B-parameter model like Flux won't fit in your VRAM at fp16 or even fp8, GGUF is how most people actually run it anyway. load_GGUF is cardenluo's ComfyUI-Apt_Preset node for pulling a quantized .gguf diffusion checkpoint into your graph - the loader half of that story, no dequantization math for you to think about.

What it does

GGUF started life in the llama.cpp world as a single-file container for quantized LLM weights, and jumped to diffusion models in August 2024 when city96's ComfyUI-GGUF node pack brought it over - two weeks after Flux's 12B parameters priced a lot of consumer GPUs out of running it at full precision. load_GGUF is this pack's own take on the same job: point it at a .gguf file and it hands back a MODEL you can wire straight into a KSampler like any other checkpoint.

It's deliberately narrow - one required input, one output, no bells. If you want GGUF folded into the pack's bigger all-in-one checkpoint loader (which also handles SD/XL/Flux/Wan/SD3.5/Hi-Dream combos, override slots, and saved presets), that's a separate node - Sum_load_adv - mentioned in the pack's README but not covered here. load_GGUF is the plain, single-purpose version.

Inputs and outputs

• unet_name - a dropdown populated from the GGUF files you have in your models folder (empty until you actually have one there).

Output: MODEL.

The quantization ladder, briefly

Since this node's whole job is picking a quant level, it's worth knowing what the levels actually mean before you download a multi-gigabyte file blind. Q8 is essentially indistinguishable from fp16 at half the size - take it whenever it fits. Q6 and Q5 are still close to lossless; Q5 is generally considered the last tier where you won't notice a difference. Q4_K_M is the accepted compromise on 12GB cards - visible loss if you go looking for it (fine detail on things like logos and text is where it shows first), but acceptable. Below Q4, quality drops off fast and, counterintuitively, can even run slower than Q8 because of dequantization overhead - lower-bit quants pay a tax that direct 8-bit hardware paths don't. If you're stacking LoRAs on top, know that GGUF gets noticeably slower with them (each layer gets dequantized, patched, and requantized on the fly); community advice when VRAM-capped is to drop a quant level to make room for the LoRA rather than fight it at a higher quant.

Installing it

Through ComfyUI Manager: search ComfyUI-Apt_Preset. Manual install:

cd ComfyUI/custom_nodes
git clone https://github.com/cardenluo/ComfyUI-Apt_Preset.git

Run install.bat (Windows) or install requirements.txt by hand, then restart.

This node has a real dependency the pack's README calls out explicitly: install ComfyUI-GGUF first. load_GGUF needs that pack's GGUF-reading machinery to actually load the format - it isn't optional the way some of this pack's other cross-references are. Grab your actual .gguf model files from HuggingFace (city96 and bullerwins maintain conversions of most popular models) and drop them in your usual UNet/diffusion-models folder.

Troubleshooting

If unet_name shows an empty dropdown, that's almost always a files-not-found problem, not a bug - confirm your .gguf file is actually sitting in the folder ComfyUI scans for UNet/diffusion models, and restart ComfyUI after adding new files since the dropdown is populated at startup.

If the node itself is missing or errors on load, check two things in order: first, whether ComfyUI-GGUF is actually installed (the README's stated prerequisite), and second, whether ComfyUI-Apt_Preset imported cleanly at all - it's a large pack, and there's a documented report of it throwing IMPORT FAILED for a user, unrelated to GGUF specifically. Check your ComfyUI console at startup for the real error before assuming either dependency is the problem.

Runtime-wise, the two most common surprises are VRAM use that doesn't drop as much as expected with LoRAs attached (that's the dequantize-patch-requantize cycle described above, not a leak), and output that looks "slightly off" rather than clearly worse at Q6–Q8 - community testing describes this as closer to a shifted seed than genuine quality loss, so don't assume something's broken if a Q8 result differs subtly from an fp16 one at the same seed.