Get LlamaCPP Model

This is the node that actually loads a local LLM into ComfyUI as a proper MODEL object - the llama.cpp way. GGUF is the single-file quantized format from the llama.cpp world, and this node hands a .gguf file to llama-cpp-python's Llama() class and returns the loaded model for the pack's chat nodes to use. If you've been running GGUF diffusion checkpoints, this is the same format, doing the same trick for language models: Q8 is essentially fp16 at half the size, and a Q4 model lets a 7B–8B LLM live happily on modest VRAM.

The one thing to know before anything else

llama-cpp-python is not in the pack's requirements.txt. The pack ships torch, pillow, numpy, and ollama - nothing else. So pip install -r requirements.txt does not make this node work. You install it yourself:

pip install llama-cpp-python

For GPU offload you'll want the CUDA build - the node loads with n_gpu_layers=-1 (all layers on the GPU), so without a GPU build you're either CPU-only or reinstalling with CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall. This is the single most common way people hit "ModuleNotFoundError: llama_cpp" with this pack.

Inputs and output

• model_name (STRING) - the filename of the GGUF, e.g. qwen2.5-7b-instruct-q4_k_m.gguf.
• models_dir_path (STRING) - absolute path to the folder holding it.
• chat_format (STRING) - the chat template string llama.cpp recognizes (e.g. llama-3, qwen2, chatml). If you leave it empty, llama.cpp guesses - and it guesses wrong often enough that you should set it.
• n_ctx (INT, default 4094) - the context window in tokens.
• Optional: chat_handler (MODEL) - plug the pack's vision chat handler in here for VL models.
• Output: model (MODEL) - wire it into the pack's Get LLM Response.

The output type is MODEL, which is this pack's own convention for "an arbitrary Python object passed through the graph" - it's not a diffusion model, so don't try to plug it into a KSampler. It only mates with other nodes in this pack (Get LLM Response, and the chat handler producer).

Install and failure modes

Pack install is standard (ComfyUI Manager → "Sagado Nodes for ComfyUI", or clone into custom_nodes), and then the extra llama-cpp-python step above. If you get Model not found at: <path>, the file name or directory is wrong - the node checks the path before loading. If loading succeeds but inference is absurdly slow, you're on the CPU build. And mind n_ctx: a big context window on a quantized model costs real RAM/VRAM, so 4094 is a sane default; don't crank it to 128k and wonder why you OOM'd.