LLM GGUF Model Loader

LLMGGUFModelLoader is the version of the model loader that lets you run this whole "LLM as SDXL text encoder" idea on a card that can't hold a full-precision model. It loads the language model from a GGUF file - llama.cpp's single-file quantized format - sitting in your ComfyUI/models/llm/ folder. Same outputs as the plain loader (LLM_MODEL, LLM_TOKENIZER, info), same job, roughly a third of the VRAM.

GGUF is the format that made 12B+ models runnable on consumer cards, and the logic transfers cleanly here. Gemma-3-1b is a 1B model, so it's not enormous to begin with, but it still has to share your GPU with an SDXL checkpoint, an adapter, and the sampler's working memory. Quantizing the encoder is exactly the "squeeze the encoder, not the diffusion model" move the community converged on for every LLM-encoded path.

What it actually does

The loader is honest about being a thin wrapper around Transformers. It calls AutoModelForCausalLM.from_pretrained(model_path, gguf_file=model_name, ...) with output_hidden_states=True and bf16, because the encoder needs those hidden states (the LLMTextEncoder node eats them). Two knobs to know:

• device - auto (the default) picks your GPU if torch sees one, otherwise CPU. You can pin cuda:0, cuda:1, or cpu explicitly.
• force_reload - the loader caches the loaded model and only reloads when the file changes. Flip this on when you swap GGUF files mid-session, or you'll keep encoding with the old one.

Outputs: model, tokenizer, and an info string that reports the path, device, and whether it actually loaded.

The gotcha nobody warns you about

The tokenizer isn't loaded from your GGUF file. It's hardcoded to unsloth/gemma-3-1b-it via AutoTokenizer.from_pretrained. That works great when you have internet or that model is already in your HuggingFace cache - it's the non-gated mirror of the model the README recommends anyway. It also means the node dies offline if that tokenizer was never downloaded. So: either accept that this loader needs a one-time tokenizer fetch, or use LLMModelLoader with a fully local gemma-3-1b-it folder instead.

One more thing worth knowing: the trained adapter was fit on the full model's hidden states. A quantized GGUF shifts those states a little, and "a little" can be a visible quality change in the generated image - the community dispute about how much quantizing an encoder costs runs from "completely fine" to "big difference." Q8 basically matches fp16; go lower than Q4 and you're gambling on your own eyes. If the results look off, compare against the full-precision loader before you blame your prompts.

Install

This is part of the ComfyUI LLM SDXL Adapter pack - Manager search "ComfyUI LLM SDXL Adapter", or:

cd ComfyUI/custom_nodes/
git clone https://github.com/NeuroSenko/ComfyUI_LLM_SDXL_Adapter.git

Restart ComfyUI. GGUF support needs the gguf>=0.17.1 dependency from the pack's requirements - that's the only extra this loader drags in. Drop your .gguf file into ComfyUI/models/llm/ and it appears in the model_name dropdown automatically.

Troubleshooting

• Model doesn't show in the dropdown: the node scans ComfyUI/models/llm/ for .gguf files. Wrong folder, no dropdown. ComfyUI/models/LLM/ (capitalized) also works as a fallback.
• Tokenzier/network error on load: the hardcoded unsloth/gemma-3-1b-it tokenizer fetch. Let it download once, or pre-cache it with huggingface-cli download unsloth/gemma-3-1b-it.
• Suddenly different images after switching quant: expected. Treat a quant swap like a seed change and re-check the output before committing to it.