GGUF Loader (Advanced)

The plain GGUF Loader has exactly one setting: which file to load. This one is what you reach for when that's not enough - when a LoRA on top of a quantized model is choking your VRAM or your generation speed, and you actually need to control how the model gets dequantized instead of just accepting whatever the default does.

The problem it's solving

Applying a LoRA to a GGUF model isn't free. Each patched layer has to be dequantized to a usable precision, have the LoRA's adjustment applied, and get requantized - every single sampling step. That cycle is the whole reason "GGUF + LoRA" runs noticeably slower than "GGUF alone," and it's also where you're most likely to run out of VRAM if the dequant precision is higher than it needs to be. The Advanced loader exposes exactly the dials that control that cost.

Inputs and outputs

• gguf_name - the model file, same as the plain loader.
• dequant_dtype (default default, or target / float32 / float16 / bfloat16) - what precision the compressed weights get unpacked to before compute. Leaving it on default lets the node pick sensibly; forcing it to something like float16 can help on a card that's struggling with the default choice.
• patch_dtype (same options) - the precision used specifically when a LoRA patches a layer. This is the one that actually addresses the dequant-patch-requantize cost - a leaner patch_dtype means less work and less memory per patched layer.
• patch_on_device (boolean, default false) - whether LoRA patching happens on GPU (true, faster, costs more VRAM) or gets handled off-GPU (false, slower, saves VRAM). This is the single biggest lever if you're OOMing specifically when a LoRA is attached.

Output is MODEL, same as the plain loader - it drops into your graph the same way, just with more you can tune before it gets there.

Installing it

Same pack, same steps as every node here: ComfyUI Manager → search gguf → install → restart, or

cd ComfyUI/custom_nodes
git clone https://github.com/calcuis/gguf

No extra dependencies on the current version. Model files go in ComfyUI/models/diffusion_models, exactly like the plain loader - this node reads the same files, it just gives you knobs on top.

When to actually use this over the plain loader

Honestly: don't, until something's wrong. Start with plain GGUF Loader and its defaults. Switch to this one specifically when you hit one of two situations - a dtype-related error (something complaining about a mismatched precision, usually surfacing when a LoRA is in the graph), or you're OOMing on a LoRA'd GGUF run that would otherwise fit. This node isn't a better version of the basic loader for everyday use; it's a troubleshooting tool with more surface area to misconfigure if you don't need it.

Common issues

LoRA + GGUF is still slow even after tuning these. That's expected to some degree - the dequantize-patch-requantize cycle has an inherent cost that these knobs can shrink but not eliminate. If speed genuinely matters more than the VRAM headroom, consider dropping a quant tier on the base model (a Q4 model with room to spare for LoRA patching can outrun a Q8 model that's fighting for every byte) rather than only tuning dtypes here.

patch_on_device=true OOMs. That's the trade you asked for - GPU-side patching is faster but costs VRAM. Flip it back to false if you're VRAM-limited; you'll trade some speed for headroom.

Not sure which dtype to pick. Leave dequant_dtype on default unless you have a specific error pointing at it. This node rewards changing one setting at a time and comparing, not guessing at a full combination up front.