Unet Loader (GGUF)

This is the node that lets you run Flux on a card that can't actually hold Flux. Same deal for SD3.5, Wan, Hunyuan, Z-Image - any of the big transformer models that laugh at your 8 or 12GB of VRAM. You point it at a .gguf file, it hands back a MODEL, and you carry on like nothing happened. It's a straight swap for ComfyUI's stock Load Diffusion Model node, and for a lot of people it's the only reason these models run locally at all.

Why you'd reach for it

When Flux dropped in August 2024 with 12 billion parameters, the full fp16 stack just didn't fit on consumer cards. GGUF is the fix. It came out of the llama.cpp world as a single-file container for quantized weights, and city96's node pack - the one this node lives in - brought it over to diffusion about two weeks after Flux launched. The reason it works: transformer/DiT models tolerate aggressive quantization in a way the old conv-based UNets never did, so you can shave a 12B model down to a few bits per weight and the output barely flinches.

The whole point is the ladder. .gguf files come in tiers, and picking the right one is the actual skill:

• Q8 is basically fp16 at half the size. If it fits, just use it - the difference is like a slightly nudged seed.
• Q5 / Q6 are the last stops before you can see anything at all.
• Q4_K_M is the accepted compromise for 12GB cards. A touch soft, totally usable.
• Q3 / Q2 are for the genuinely desperate (4–6GB). Real quality loss, and counterintuitively they can run slower than Q8, because the lowest quants pay a dequantization penalty that Q8 avoids.

Where do you actually see the damage? Not overall vibes - specifics. Prompt something holding an iPhone and watch a low quant fail to draw the logo. Text and fine detail go first.

The inputs and outputs that matter

There's exactly one input worth discussing, because there's only one: unet_name, a dropdown of the .gguf files it finds in your ComfyUI/models/unet folder. Pick your quant, done. The output is a single MODEL, the standard ComfyUI type - wire it straight into your KSampler, or through a LoRA loader first, exactly where "Load Diffusion Model" would've gone.

One thing beginners miss: a GGUF loaded here is the diffusion model only. It doesn't carry a VAE or text encoder the way a full checkpoint does, so you still bring those separately. On a tight card, quantize the text encoder too - the encoder is often the real bottleneck, not the model - using the pack's separate *CLIPLoader (gguf) nodes, not this one. You'll find this node under the bootleg category, which is city96 being cheeky, not a warning.

Installing it

Easiest path: ComfyUI Manager - search "ComfyUI-GGUF", install, restart. By hand it's barely harder:

cd ComfyUI/custom_nodes
git clone https://github.com/city96/ComfyUI-GGUF
pip install --upgrade gguf

Then restart ComfyUI. The gguf pip package is the only dependency. On a portable Windows build you clone into ComfyUI/custom_nodes and run python_embeded\python.exe -s -m pip install -r requirements.txt against the embedded Python instead.

Then grab a model. city96 hosts pre-quantized conversions on HuggingFace - flux1-dev, flux1-schnell, stable-diffusion-3.5-large - and between city96 and bullerwins the community ships quant packs for most new releases within a day. Drop the .gguf into ComfyUI/models/unet, refresh, and it shows up in the dropdown.

Where people get burned

• Empty dropdown / wrong folder. The file goes in models/unet, not models/checkpoints. That one trips people up constantly. Refresh or restart after adding it.
• "Custom ops" errors on load. Your ComfyUI is too old to load a UNET-only file. Update it - the README flags this explicitly.
• LoRAs get slow. They work (through the built-in loaders, flagged experimental), but each layer has to be dequantized, patched, and requantized, which tanks your speed. If you're VRAM-capped, the counterintuitive fix is to drop a quant level to make room - Q4 with a LoRA beats Q8 without.
• You installed "Force/Set CLIP Device." That node is not part of this pack. Don't install it if you have one GPU, and definitely don't set it to cuda:0 and then complain about OOM.
• You should've just used fp8. On a 16GB+ card where the fp8 file fits, fp8 is simpler, a touch faster, and holds its speed with LoRAs. GGUF's advantage starts where fp8 stops fitting. (macOS Sequoia note: stick to torch 2.4.1 - the 2.6 nightlies throw an "M1 buffer is not large enough" error.)