LLM Loader (GGUF)

Every text-generation node in gokayfem's VLM Nodes - LLMPromptGenerator, LLMSampler, ChatMusician, the structured output nodes - has one thing in common: they all want a CUSTOM model handle, and LLMLoader is what produces it. Think of it as the pack's version of a checkpoint loader, but for GGUF language models. Drop any GGUF into the right folder, pick it here, and hand the resulting model wire to whichever text node you need.

What it's for

If you're running any of the pack's local LLM features, you start here. It's the load-bearing node for the whole "LLM (local GGUF)" section of the pack, and it's also how you use a LLaVA GGUF as a plain language model - the README notes you can even run LLaVA models through it as text-only LLMs.

How it works

It loads a GGUF through llama.cpp into a reusable handle. The ckpt_name dropdown lists every file in your models/LLavacheckpoints folder (the pack registers that folder on import), so no special install location - just drop the GGUF in and restart ComfyUI. The current version queries your installed llama.cpp build for its real capabilities instead of guessing, and it includes the modern runtime options - n_batch/n_ubatch batching, flash-attention policy, mmap, multi-GPU layer splitting - as optional inputs that default to sensible values.

The inputs that matter

• ckpt_name - the GGUF file. Any llama.cpp-compatible model.
• max_ctx - context length; 2048 default is fine for short prompt work, raise for long chats.
• gpu_layers - how many layers to offload to GPU. Default 27 suits a 7B model; set it to the max your VRAM allows for speed.
• n_threads - CPU threads; only matters for the layers that stay on CPU.

Output is a single CUSTOM model handle. Feed it to any of the pack's sampler/prompt nodes.

Install

The pack install is the same as ever:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
python -m pip install -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements.txt

The llama.cpp part is the bit that needs attention: this node is the pack's most-Googled install failure. The old versions auto-downloaded llama-cpp-python wheels with a script that could 404 or land on a wrong-wheel filename (there's a whole Reddit thread history of Windows users fixing a metal-cp312 wheel that doesn't exist). Current versions moved llama.cpp to an optional extra - install a llama-cpp-python wheel built for your backend, per the README's link to the official install docs.

The one gotcha

If your model loads but everything runs on CPU, gpu_layers is probably fine - the problem is a CPU-only llama.cpp build. Reinstall with the right backend flag and the node will offload properly. And remember: the loader itself is lazy - it hands back a handle and only actually loads the model when a downstream node executes, which is why a wrong-path error can show up at the sampler instead of here.