Unet Loader (Dynamic VRAM)

If you've got a .gguf file sitting in your models/unet folder, this is the node that actually reads it. Unet Loader (Dynamic VRAM) is the flagship of the molbal/ComfyUI-GGUF pack - a dropdown of every GGUF in your unet folder, one MODEL output, wire it straight into a KSampler. That's the whole job, and it's the reason a 12B transformer like Flux, SD3.5, or Krea 2 will run on a card that can't hold the fp16 weights at all.

Quantization is the point. GGUF stores weights at 4–8 bits instead of 16, and on transformer/DiT architectures the quality hit is shockingly small - Q8 is basically fp16 at half the size, and Q4 is the accepted compromise for 12GB cards. Old conv-based UNets degraded badly under quantization, which is why this whole format didn't exist until the Flux generation. The version of ComfyUI-GGUF you have now is molbal's maintained fork, which extends the original city96 pack to the models people actually run today: Flux, SD3.5, Krea 2, Ideogram 4, LTX 2.5.

How it works

The loader reads the single mmap-friendly GGUF file, then looks at its quant mode and picks the right execution path. If the file is Q8_CR - this fork's custom INT8-ConvRot format - the weights stay INT8 and run through ComfyUI's native INT8 backend instead of being dequantized. Standard GGML quants (Q8_0, Q4_0, ...) go through the regular GGML ops. Either way you get the same MODEL object you'd get from a full-precision loader, just smaller in memory. The "Dynamic VRAM" variant wraps that in ComfyUI's aimdo memory manager, which tries to use as close to 100% of your VRAM as possible without OOM and unloads cleanly to disk instead of page-swapping.

The inputs that matter

There's exactly one: unet_name, a dropdown of .gguf files in models/unet (the pack also maps models/diffusion_models, so either works). Drop your file there, refresh the dropdown, pick it. The output is MODEL, which feeds your KSampler or CFGGuider exactly like any other checkpoint loader.

Installing

Search ComfyUI-GGUF in the ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/molbal/ComfyUI-GGUF

Restart ComfyUI, then make sure the gguf package is present (pip install --upgrade gguf - it's the only hard inference dependency; sentencepiece and protobuf are optional extras for tokenizers). You also need ComfyUI v0.27.0 or later, because that's the build that ships the native INT8 path this fork leans on.

Where people get burned

• The Dynamic VRAM nodes demand a compatible ComfyUI. If you launched with --disable-dynamic-vram (or your build predates it), you'll get a clear error saying so. Note that dynamic VRAM doesn't play with --reserve-vram, and on low-RAM rigs it can get aggressive about unloading models between runs - if you're seeing constant reload stalls, that's the feature doing its thing.
• No _K quants on diffusion models. Q4_K_M and friends are text-encoder-only here. They may load or they may not, and inference can crawl. Use the flat Q8_0/Q5_*/Q4_0 ladder instead.
• LoRAs slow GGUF down. Every layer has to dequantize, get patched, and requantize. If you're LoRA-heavy and VRAM-capped, dropping a quant level to make room beats fighting it.
• MacOS Sequoia: torch 2.4.1 is required; newer 2.6.x nightlies hit an "M1 buffer is not large enough" error.