GGUFLoaderKJ

GGUF is how most people run the big models now. It came out of the llama.cpp world and jumped to diffusion right when Flux's 12B parameters priced fp16 off consumer cards; the Q-ladder (Q4 through Q8) is the whole point, and the rule of thumb is settled: Q8 is basically indistinguishable from fp16 at half the size, so if it fits, just use it, with Q4_K_M the accepted compromise on a 12GB card. This node loads one of those .gguf files, but with more control than the stock GGUF loader gives you.

Important: this node requires city96's ComfyUI-GGUF to be installed. KJNodes doesn't read GGUF on its own, it builds on top of city96's pack, which is the thing that actually does the loading. If you don't have that, this node won't work.

How it works

It loads a quantized GGUF checkpoint and outputs a MODEL, same as any loader, but exposes advanced options that the basic loader hides. The extra knobs mostly deal with the tax GGUF pays: dequantization overhead, and the extra cost when LoRAs force a per-layer dequantize-patch-requantize cycle. For most people the defaults are fine and the value is just picking the file; the advanced controls are there for when you're squeezing a specific card.

The inputs and outputs that matter

• model_name - the GGUF file to load, picked from your GGUF models folder. This is the only one most people set. Choose your quant here (Q8 if it fits, Q5/Q4_K_M if you're tighter on VRAM).
• extra_model_name (default none) - an optional second GGUF, for setups that load an additional weights file alongside the main one.
• patch_on_device (default off) - whether to apply patches (LoRAs) on the GPU. On can be faster if you have the VRAM headroom; off keeps them off the card.
• enable_fp16_accumulation (default off) - an accumulation-precision option that can speed things up on supported hardware. Leave it off unless you know your setup benefits.
• dequant_dtype, patch_dtype, attention_override (all default to sensible defaults) - low-level precision and attention controls. Don't touch these until you have a specific reason; the defaults are chosen to just work.

The output is MODEL, wired into your sampler like any checkpoint.

Installing it

You need two things: KJNodes and ComfyUI-GGUF.

• ComfyUI Manager - search and install both KJNodes for ComfyUI and ComfyUI-GGUF, then restart.
• Manual:

  cd ComfyUI/custom_nodes
  git clone https://github.com/kijai/ComfyUI-KJNodes
  git clone https://github.com/city96/ComfyUI-GGUF
  pip install -r ComfyUI-KJNodes/requirements.txt
  

  then restart ComfyUI.

Your .gguf files go in the GGUF models folder (typically ComfyUI/models/unet or the diffusion-models folder that ComfyUI-GGUF reads).

Common issues

The node isn't there / errors about GGUF. ComfyUI-GGUF isn't installed. This node depends on it; install city96's pack and restart.

LoRAs slow it to a crawl. That's inherent to GGUF, not this node. Applying a LoRA makes each layer dequantize, patch, and requantize, which can tank throughput on a quantized model where fp8 wouldn't. The community fix when you're VRAM-capped: drop a quant level to make room (running Q4 with the LoRA often beats Q8 without it), or use an fp8 model instead if it fits, since fp8 holds its speed with LoRAs.

Which quant? Q8 matches fp16 and is the safe pick if it fits; Q5 is the last tier before you start seeing it; Q4_K_M is the 12GB sweet spot; Q3 and below are for genuinely desperate low-VRAM situations and it shows. Counterintuitively Q8 can be faster than Q2/Q3, because the very low quants pay a heavier dequantization penalty, so don't reach for a tiny quant expecting speed.

No .gguf files in the dropdown. They're not in the folder ComfyUI-GGUF scans. Put them where your other diffusion models live and refresh.