GGUF Diffusion Model Loader 🧠

If you just grabbed a Q4 or Q8 Flux/SD3 .gguf because your card can't hold the fp16 file, I have to pump the brakes before you wire this in and wonder why nothing got faster or smaller. Here's the sentence that explains this whole node: it fully dequantizes every tensor back to fp16 at load time, then hands the result to ComfyUI's normal diffusion-model loading path. The output is a perfectly standard MODEL that plugs into KSampler exactly like one from a stock UNETLoader - and it occupies the same full fp16 VRAM/RAM footprint as if you'd loaded the safetensors.

So the reason GGUF normally exists in ComfyUI - fitting a 12B DiT that otherwise spills off your card - is not what this node delivers. To get those memory savings you want city96's ComfyUI-GGUF extension, which keeps weights quantized on the GPU with custom ops. OmniNodes' loader is for a narrower, still-legit use: you have a .gguf on disk (a friend shared it, it's the only version of a niche model you could find), you don't want to install another extension just to open it, and you've got the VRAM for the fp16 footprint anyway. It turns an otherwise-unreadable file into a working model in one click. That's genuinely useful - it just isn't the low-VRAM lever.

How it works

It reads the file with the gguf package's GGUFReader, dequantizes every tensor via gguf.dequantize, casts to your chosen precision, and pushes the plain state dict through comfy.sd.load_diffusion_model_state_dict - ComfyUI's own core loading code. Since it goes through the same path as a normal UNet load, downstream behavior matches a safetensors model exactly. Any tensor type the gguf library can't dequantize gets skipped and reported in summary, so nothing fails silently.

Inputs and outputs

• gguf_name - dropdown of .gguf files found in the usual model folders (diffusion_models, unet, unet_gguf, checkpoints). Note the unet_gguf there: it also sees files you've placed for ComfyUI-GGUF. If the dropdown is empty, drop the file into one of those folders or use path_override with an absolute path.
• weight_dtype - fp16 (default) or fp32. fp16 halves RAM/VRAM versus fp32 at negligible precision cost on modern GPUs; stick with the default unless you have a specific reason to go fp32.

Outputs: model (a normal MODEL → wire into KSampler), and summary (a string telling you how many tensors were dequantized and whether any were skipped). The summary's last line is a useful honesty check: it reminds you this was a full-precision load, not a memory-reduced one.

Installing

OmniNodes is TensorVizion's large all-purpose pack (audio, image, latent, model utilities, GGUF, sampling, video, web, workflow categories), and GGUF is its newest section. Install once, add the one extra dependency:

cd ComfyUI/custom_nodes/
git clone https://github.com/TensorVizion/OmniNodes
cd OmniNodes && pip install -r requirements.txt

Then restart ComfyUI. Or use ComfyUI Manager and search "OmniNodes" - either way you still need pip install gguf into the same Python environment ComfyUI runs in, because it's not bundled. Find the node under TensorVizion/GGUF in the search menu.

Troubleshooting

Missing gguf? The node returns a clear [TensorVizion] ... Run: pip install gguf message instead of crashing - install it and restart. "File not found" means either your file isn't in a folder ComfyUI scans or you need path_override. The other common failure is ComfyUI's core loader rejecting the dequantized state dict, which usually means the architecture isn't recognizable from key names alone - check that you actually downloaded a diffusion-model GGUF and not, say, a text encoder or an LLM (run GGUFFileInfo on it first). And if your ComfyUI is old enough to lack load_diffusion_model_state_dict, the node says so plainly: update ComfyUI. The decision you should make before all of that, though, is the VRAM one at the top of this page.