Unet Loader (GGUF/Advanced)

This is the node that actually loads the model - the diffusion transformer itself, the big one - from a quantized .gguf file. It's the whole reason city96's pack exists. If you've ever seen a Flux, SD3.5, or Wan workflow running on a card that has no business running it, this (or its plain sibling) is doing the heavy lifting. The "Advanced" in the name means it exposes a few extra dtype controls that the regular Unet Loader (GGUF) hides.

Why load a UNET from GGUF at all

When Flux dropped in August 2024 with 12B parameters, the full-precision stack simply didn't fit on consumer cards. city96's insight, borrowed from the llama.cpp world, was that transformer/DiT models tolerate quantization in a way the old conv-based UNets never did - so you could compress those weights down a variable-bitrate ladder and run the model at all. That's the payoff: a Q4_K_M Flux on a 12GB card, a Q5_K_M Wan 14B on a 10GB card. The README's own instruction is blunt: just replace the stock "Load Diffusion Model" node with this one and point it at a .gguf file. Everything downstream stays the same.

A quick reality check before you reach for it, though. Quantization is for when the model genuinely doesn't fit. If an fp8 version fits your card, fp8 is simpler, slightly faster, and holds its speed when you stack LoRAs - GGUF's advantage begins exactly where fp8 stops fitting. And on a card that "barely fits" a model, the real speedup sometimes comes from dropping --lowvram flags you didn't need, not from quantizing harder. Try the simple thing first.

The inputs - and when to touch the advanced ones

• unet_name - the .gguf diffusion model, read from ComfyUI/models/unet. This is the input you'll set every time.
• dequant_dtype - the precision the weights are dequantized to during inference (default, target, float32, float16, bfloat16). Leave it on default unless you're chasing a specific precision issue.
• patch_dtype - the dtype used when patching, i.e. when a LoRA gets applied. Same options, same advice: default is right almost always.
• patch_on_device - a boolean. When on, LoRA patching happens on the GPU instead of in system RAM. It can speed up LoRA application at the cost of a little extra VRAM during the patch. Off by default.

That's the entire difference between this and the plain loader: those three dtype/device knobs. If you don't know why you'd change them, you don't need to - the regular Unet Loader (GGUF) is the same node with sane defaults baked in. The single output is MODEL, which wires into your KSampler exactly like any other diffusion-model loader.

Installing it

It's part of city96's ComfyUI-GGUF pack. ComfyUI Manager: search "ComfyUI-GGUF", install, restart. Manual: cd ComfyUI/custom_nodes && git clone https://github.com/city96/ComfyUI-GGUF, then pip install --upgrade gguf, then restart. On Windows portable, clone into ComfyUI/custom_nodes/ComfyUI-GGUF and run the embedded Python against the pack's requirements.txt. The gguf library is the only dependency.

Put your model files in ComfyUI/models/unet. city96 hosts pre-quantized Flux, SD3.5, and other models on HuggingFace (FLUX.1-dev-gguf, stable-diffusion-3.5-large-gguf, and friends), and quant packs for new models now typically appear within a day of release. The node shows up under the bootleg category.

Common issues

The README flags one directly: make sure your ComfyUI is recent enough to support custom ops for UNET-only loading - an out-of-date install is a common cause of load failures here. The advanced knobs are the other trap: people crank dequant_dtype to float32 expecting "better quality" and just burn VRAM for no visible gain, so change them only with a specific reason. And the LoRA tax is real - when you apply a LoRA, GGUF dequantizes, patches, and requantizes each layer, which can slow generation noticeably; patch_on_device helps a little, but if you're capped, the community move is to drop a quant level to make room rather than fight the overhead. macOS Sequoia users need torch 2.4.1 (the pack's issue #107 has the details).