llama-cpp

Most of NodeGPT's LLM provider nodes don't run a model at all - they point at some external server (LM Studio, Ollama, oobabooga) that does the heavy lifting. Not this one. The llama-cpp node loads a GGUF quantized model directly in-process via llama.cpp, which means it's the one "everything is local, no server, no API key" option in the pack. The README even annotates it in parentheses: llama-cpp (slow). That's the author being honest with you, and you should believe it.

The mechanism is exactly what you'd expect: llama-cpp-python loads a .gguf file into memory, and the node hands AutoGen a config pointing at that in-process model instead of a network endpoint. It's the same architecture the pack uses for the llava vision node (which also loads GGUF files), just text-only.

What you set

• Model - a string, defaulting to dolphin-2.1-mistral-7b.Q5_K_M.gguf. That tells you the pack expected you to grab a Mistral-7B-class GGUF and drop it in the pack's models folder. The Q5_K_M quantization is a solid middle ground - decent quality, manageable size.
• model_path - optional, but you'll almost certainly need it: if your GGUF lives somewhere other than the pack's default search location, this is how you point at it.
• n_ctx - context window, default 2048. Keep it there for small VRAM; raise it if the model starts forgetting the conversation.

Output: LLM, ready to feed any agent node. Pair it with the dolphin-mistral default and you've got a fully offline, keyless setup for the whole pack.

Where people get burned

Performance first: the README's "(slow)" is doing a lot of work. A 7B GGUF through llama.cpp with the default settings - especially on CPU - turns a snappy multi-agent chat into a slideshow, and a group chat with a 50-round budget becomes a project. Set expectations, lower max_round, and don't blame the node for physics.

Then there's the model-file hunt. The default filename is a real model (dolphin-2.1-mistral-7b.Q5_K_M.gguf is a known TheBloke-era release), but you still have to obtain and place it, and model_path must actually resolve. The README's llava section is the only place it bothers to name a model source, so for text models you're mostly on your own.

And the pack's meta-problem, one last time: as of this writing xXAdonesXx/NodeGPT returns 404 and the author's account is gone. The README's install is therefore historical:

cd ComfyUI/custom_nodes
git clone https://github.com/xXAdonesXx/NodeGPT

Restart to auto-install requirements (llama-cpp-python is the heavy one). If you just want a local GGUF inside ComfyUI in 2026, there are maintained llama-cpp node packs that will serve you far better than this fossil - but this node is a perfectly readable introduction to the concept.