🖥️LLM-GGUF Loader

If you've quantized a diffusion model before, you already understand 90% of what this node does - same GGUF format, same Q-ladder tradeoffs, just pointed at a language model instead of a checkpoint. GGUF started life in the llama.cpp world as a single-file container for quantized weights, and this node is comfyui_LLM_party's front door into that ecosystem: point it at a .gguf LLM file and it hands you a loaded model you can wire into the pack's LLM nodes.

Why bother with GGUF instead of just loading a full transformers model? Size and portability, mostly. A GGUF quant is one self-contained file, mmap-friendly, and the quality ladder is well understood at this point: Q8 is close enough to fp16 that most people can't tell the difference in a blind test, Q5 is the last stop before you start noticing it, and Q4_K_M is the accepted compromise when VRAM is tight. Below Q3 you're trading real quality for a smaller footprint, and - counterintuitively - sometimes losing speed too, since low-bit quants pay a dequantization tax that Q8 mostly avoids.

The inputs here are the standard llama.cpp knobs, not anything the party invented. model_path is the full path to your .gguf file on disk - no picker, no dropdown, you type it. max_ctx sets the context window (default 512, which is tiny - bump it to whatever your model actually supports if you're doing anything beyond a one-liner). gpu_layers controls how many transformer layers get offloaded to GPU; higher means faster but more VRAM, and if you're running CPU-only you'd drop this toward 0. n_threads is the CPU thread count for whatever isn't offloaded. is_locked (on by default) keeps the model pinned in memory so re-running the graph doesn't reload it from disk every time - leave it on unless you're actively swapping models, in which case turn it off and route through the pack's Clear Model node between swaps. The single output, model, plugs straight into the LLM Party's local-model nodes (set model_type to LLM-GGUF on the node you connect it to).

Installing it. This node ships inside comfyui_LLM_party as a whole - there's no separate install. Get the pack via ComfyUI Manager (search "comfyui_LLM_party") or git clone https://github.com/heshengtao/comfyui_LLM_party into custom_nodes, then restart. The step people skip is the one right after: cd into the pack folder and run pip install -r requirements.txt inside ComfyUI's own Python environment - on a portable/embedded install that means explicitly calling python_embeded\python.exe -m pip install -r requirements.txt, not whatever pip happens to be on your system PATH.

The real gotcha is llama-cpp-python. It's compiled code, and this loader depends on it directly. If your GGUF model throws a load error, the README's own advice is to grab the latest wheel from abetlen/llama-cpp-python releases rather than assume pip install grabbed something CUDA-compatible for your card. If you don't actually need GGUF models, config.ini has a fast_installed flag you can set to true specifically to skip this dependency during setup - a sign of how often it's the thing that breaks a fresh install. Get the balance of gpu_layers wrong for your VRAM and you'll see an OOM or a silent fallback to CPU-only speeds rather than a clean error, so if generation feels glacial, that's the first knob to check before blaming the model itself.