Nodes/ComfyUI_Dif_GGUF/DualCLIP_GGUF_Loader
ComfyUI Node

DualCLIP_GGUF_Loader

The GGUF loader for Flux, SDXL and friends

By smthemex·Created about a month ago·Updated 5 days ago· 18
DualCLIP_GGUF_Loader
    • CLIP
    gguf1
    gguf2
    type
    device

    Somewhere between SDXL and Flux, "the text encoder" became plural. SDXL runs CLIP-L and OpenCLIP-G, Flux runs CLIP-L and T5-XXL, and SD3, Hunyuan Video and LTXV all pair up two encoders too. If you're quantizing the diffusion model to fit your card but still loading those two encoders at full fp16, you're leaving the second half of the savings on the table. DualCLIP_GGUF_Loader from smthemex's ComfyUI_Dif_GGUF pack loads the pair in one shot, with both files as .gguf.

    The idea mirrors the single-file CLIP_GGUF_Loader from the same pack, and the wider GGUF movement city96 kicked off for Flux: quantized weights stay quantized and are dequantized lazily at forward time, so you never inflate them back into fp16 in VRAM. T5-XXL is 4.7B parameters on its own; on a 12GB card, running it at Q6 instead of fp16 is the difference between a workflow that fits and one that OOMs halfway through the first batch.

    How it works

    Each .gguf is read and its llama.cpp-style tensor names are remapped to what ComfyUI's T5/CLIP/llama implementations expect, then both state dicts are handed to ComfyUI's text-encoder loader together. The pack forces a sane compute dtype along the way, because GGUF norm tensors come through as F32 and would otherwise drag the whole encoder to float32 compute. The output is a single CLIP object holding both encoders - exactly what the model's conditioning expects - wired straight into CLIP Text Encode (Prompt).

    The inputs that matter

    • gguf1 and gguf2 - the two encoder files, picked from the same dropdown (files in your text_encoders, clip_gguf, and gguf model folders). The dropdown only offers .gguf files, so in practice both ends get quantized. "none" appears in the list but the loader refuses it - fill both slots with real files.
    • type - this is the field to get right. It defaults to flux (CLIP-L + T5-XXL), and the rest of the list tracks the dual-encoder models: sdxl, sd3, hunyuan_video, ltxv, hidream, kandinsky5, and more. Pick the architecture that matches your checkpoint; a mismatch won't error so much as silently mis-condition.
    • device - default or cpu, if you want to park the encoder weights in system RAM instead of VRAM.

    Installing it

    Same story as the rest of the pack. In ComfyUI Manager, search ComfyUI_Dif_GGUF, or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/smthemex/ComfyUI_Dif_GGUF
    cd ComfyUI_Dif_GGUF
    pip install -r requirements.txt
    

    The only real dependency beyond PyTorch is the gguf Python package - if you already have city96's ComfyUI-GGUF installed, it's there and you can skip the pip step. Drop your two encoder .gguf files in ComfyUI/models/text_encoders/ or models/gguf/, restart, done.

    Where people get burned

    • The obvious trap: both slots must be filled, and the type must describe the pair. Flux users pick flux and grab a quantized T5 plus a CLIP-L; SDXL users pick sdxl and grab the two CLIP variants. The default flux catches people who don't think to change it.
    • Missing gguf - if you skip requirements and run no other GGUF pack, you get ModuleNotFoundError: gguf at startup.
    • This pack needs a recent ComfyUI. It's built on the newer node API (comfy_api.latest, the extension entry-point system), so on an old install the nodes simply won't show up. Update ComfyUI before chasing anything else.

    One thing worth knowing before you pick your quant levels: the encoders are the forgiving half of the equation. Q8 is basically fp16 at half the size, and even Q4-Q6 on a T5 or a CLIP costs you far less visually than quantizing the diffusion transformer. Spend your precision on the main model, squeeze the encoders, and let both load through this node.

    Categorymodel/loaders

    Inputs (4)

    NameTypeDefaultDescription
    gguf1COMBO1 options: none
    gguf2COMBO1 options: none
    typeCOMBO12 options: sdxl, sd3, flux, hunyuan_video, hidream, hunyuan_image, +6
    deviceoptCOMBO2 options: default, cpu

    Outputs (1)

    NameTypeDescription
    CLIPCLIP