GGUF Loader

If your VRAM can't hold a model at full precision, GGUF is how you fix that without buying a new GPU - and GGUF Loader is the node that actually reads the file. Drop a .gguf diffusion model in, get a MODEL out. It's the entry point for running Flux, Wan, HiDream, and every other big model that's outgrown consumer cards, at whatever quant level your GPU can handle.

Why it exists

Full-precision weights (fp16/bf16) for anything Flux-class and up just don't fit on a 8-12GB card. GGUF compresses those weights down to 4-8 bit representations - the same trick llama.cpp uses for local LLMs, borrowed for diffusion once Flux's 12B parameters priced out most consumer VRAM. The quality ladder is well-worn territory at this point: Q8 is basically indistinguishable from fp16, Q6 and Q5 lose essentially nothing, Q4_K_M is the accepted compromise once you're squeezed onto a 12GB card, and Q3 and below are for genuinely desperate setups. If a model fits in fp8 already, that's usually simpler and just as good - GGUF earns its keep specifically where fp8 stops fitting.

How it works

The loader reads the gguf file's tensor index and dequantizes weights on the fly as sampling needs them, rather than unpacking the whole model to full precision up front. That's the entire point of the format - most of the model sits compressed in VRAM, and only what's actively being computed gets expanded. It's why a Q4 quant of a 12B model can run on a card that couldn't touch the fp16 version.

Inputs and outputs

There's exactly one input that matters: gguf_name, a dropdown of .gguf files it finds in your models folder. No dials, no toggles - you pick the file, that's the whole configuration surface. It outputs MODEL, which wires straight into your KSampler the same way any other model loader's output would.

Because GGUF-format diffusion checkpoints are almost always split files rather than a single all-in-one checkpoint, you'll pair this with a separate CLIP loader and VAE loader from the same pack (or the plain ComfyUI ones, if your CLIP/VAE aren't quantized) to build the full pipeline.

Installing it

Through ComfyUI Manager: search gguf, install, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/calcuis/gguf

then restart ComfyUI. As of the current release the gguf-connector engine that does the actual decoding ships bundled with the node itself, so there's no separate pip install and no extra dependency step - the author's own words are "no dependencies needed right away."

Drop your .gguf diffusion model files into ComfyUI/models/diffusion_models. If the gguf_name dropdown comes up empty after install, that's almost always the file sitting in the wrong folder - it won't show up under the regular checkpoints list, and it won't show up here either if it's in the wrong place.

One thing worth knowing going in: this isn't the only GGUF loader in the ComfyUI ecosystem. city96's ComfyUI-GGUF is the much more widely used pack and most GGUF tutorials online are written against it. calcuis's own README is upfront that this one is a separate, natively-coded implementation - inspired by city96's work, not a repackage of it, and the two coexist fine if you have both installed. Just know which one a workflow or tutorial is actually referring to before you go hunting for a node that isn't there.

Common issues

Empty dropdown. The file isn't in diffusion_models, or ComfyUI needs a restart (or a "refresh" of the node list) to pick up a file you just added - this node reads the folder at load time, not live.

Slow with LoRAs. GGUF's dequantize-then-patch-then-requantize cycle is inherently slower once you stack a LoRA on top, especially at higher quant levels. If a LoRA'd run feels sluggish, that's expected behavior rather than a bug - dropping a quant tier (say Q8 down to Q4) sometimes runs faster with a LoRA attached than a higher quant does, because there's less to dequantize per layer.

Wrong node for the workflow. If someone's shared workflow references a "GGUF Loader" and yours won't load it, check whether they built it against city96's pack instead - the node names are similar enough to cause real confusion, but they aren't interchangeable drop-ins for each other's workflow JSON.