ComfyUI GGUF (molbal's fork)

ComfyUI-GGUF

GGUF Quantization support for native ComfyUI models including the custom Q8_CR.

[!NOTE]
This is a fork of the original nodes, updated to support loading Ideogram 4 GGUFs and Krea 2 GGUFs. To use this maintained fork, clone https://github.com/molbal/ComfyUI-GGUF.

While quantization was previously unfeasible for regular UNET models (conv2d), transformer/DiT models such as flux are less affected by quantization. This allows running them in lower bits per weight variable bitrate quants on GPUs with less VRAM.

More details on how to use it, pre-converted models, and sample workflows are here: Documentation

For technical details on the custom Q8_CR and Q4_CR formats , memory-mapped loading, please see ARCHITECTURE.md.

Installation

[!IMPORTANT]
Make sure your ComfyUI is on v0.27.0 or later.

To install the custom node normally, git clone this repository into your custom nodes folder (ComfyUI/custom_nodes) and restart ComfyUI.

git clone https://github.com/molbal/ComfyUI-GGUF

Usage

Simply use the GGUF Unet loader found under the bootleg category. Place the .gguf model files in your ComfyUI/models/unet folder.

Pre-quantized models (🍴 icon on ones added by this fork):

• flux1-dev GGUF
• flux1-schnell GGUF
• stable-diffusion-3.5-large GGUF
• stable-diffusion-3.5-large-turbo GGUF
• Krea 2 (Both Turbo and Raw) 🍴
• Ideogram 4 🍴
• MiniMax H3 🍴
• MiniMax Music3 🍴
• LTX 2.5 🍴

[!IMPORTANT] > Please note, that this fork does not support _K quants on diffusion models, only on text encoders. They may or may not load, but inference speed may be very slow. There may be other forks, or other custom nodes with better support for these quantization types.

Initial support for quantizing T5 has also been added recently, these can be used using the various *CLIPLoader (gguf) nodes which can be used inplace of the regular ones. For the CLIP model, use whatever model you were using before for CLIP. The loader can handle both types of files - gguf and regular safetensors/bin.

• t5_v1.1-xxl GGUF
• Qwen3-VL-4B-Instruct-GGUF 🍴
• Qwen3-VL-32B-Instruct-GGUF 🍴
• Qwen3-VL-32B-Instruct-MiniMax-H3 pruned GGUFs 🍴
• Qwen3.5 GGUF text encoders (0.8B, 2B, 4B, 9B, and 27B) with a ComfyUI build containing Qwen3.5 TE support. Place the matching mmproj-*.gguf beside the text encoder for image conditioning; text-only workflows do not need it. 🍴
• Gemma 4 GGUF text encoders (E2B, E4B, 12B, and 31B) with ComfyUI v0.30.0 or later. 🍴

Converting Models (Krea 2, Ideogram 4, MiniMax H3, MiniMax Music 3)

This node pack includes a GGUF converter. It has 3 possible interfaces that you can use:

• a python file you can call directly
• a web interface
• a custom node

Each option is documented here: Quantizing models

Supported Conversion Formats

| Format | Storage / execution | Recommended use | |--------|---------------------------|-------------------------------------------------------------| | F16 | FP16 GGUF | Maximum compatibility with half-precision storage. | | BF16 | BF16 GGUF | Preserve BF16 source models where the target supports BF16. | | Q8_0 | Standard GGML 8-bit | Excellent general-quality 8-bit GGUF. | | Q5_1 | Standard GGML 5-bit | Lower storage with a quality-oriented 5-bit format. | | Q5_0 | Standard GGML 5-bit | Lower storage alternative to Q5_1. | | Q4_1 | Standard GGML 4-bit | Smaller files when VRAM or RAM is constrained. | | Q4_0 | Standard GGML 4-bit | Smallest supported format for constrained setups. | | Q8_CR | Per-row INT8 ConvRot | Maintainer recommendation for NVIDIA RTX 30-series systems. | | Q4_CR | Experimental INT4 ConvRot | Maintainer recommendation for NVIDIA RTX 30-series systems. |