Load LLM Model Advanced

Load LLM Model Basic will get you a working graph in about four clicks. This is the node you switch to when Basic stops being enough - it exposes nearly every argument from llama-cpp-python's Llama.__init__ as a widget. Which is great when you know what you're doing, and a wall of thirty inputs when you don't.

Here's the honest framing: on a typical setup you'll touch maybe five of these, leave the rest alone, and never notice. This node is the escape hatch, not the daily driver.

The inputs that actually matter

• n_gpu_layers - the big one. How many model layers get offloaded to your GPU. 0 (the default) is pure CPU inference, which is painfully slow on anything past a 1B model. Raise it to fit your VRAM; on a 12GB card you can usually offload a large chunk of a small model and leave the rest to system RAM.
• n_ctx - context window in tokens, default 512. Same as Basic, but here the pattern is explicit: it's a token budget, and it costs RAM.
• n_batch - how many tokens are evaluated per pass. Higher is faster but greedier with memory. 512 is a sane default.
• n_threads / n_threads_batch - CPU thread counts. Useful if you're doing pure CPU inference and want to pin them; otherwise ComfyUI and the OS will sort it out.
• use_mmap / use_mlock - memory-mapping (default on) lets the OS lazily page the model file instead of loading it fully; mlock pins it in RAM. If you have the RAM, use_mlock can smooth out stalls.
• seed - -1 (default) means random initialization. Fix it if you want reproducible noise for the model itself.

The rest of the list - split_mode, main_gpu, tensor_split for multi-GPU, rope_scaling_type and the whole yarn_* family for context extension, mul_mat_q, logits_all, embedding, lora_base/lora_path/lora_scale for LoRA adapters, chat_format, verbose - is a pass-through of llama.cpp options. If you don't know what YARN scaling is, you don't need it yet. The author's own documentation is the llama-cpp-python init reference, and the parameter names here match it one-for-one, so you can look anything up there.

What's disabled

Be aware that two things from that reference don't work here, per the README's known-issues list: kwargs and chat_handler. The author wired the fields he could reach through ComfyUI's node system and left the rest on the floor. Don't go hunting for a hidden chat template dropdown - there isn't one, and chat_format only takes you so far.

Setting it up

Install is the pack-standard path: ComfyUI Manager (search "ComfyUI-Llama") or

cd ComfyUI/custom_nodes
git clone https://github.com/daniel-lewis-ab/ComfyUI-Llama

with pip install llama-cpp-python if you went manual. Your .gguf models go in ComfyUI/custom_nodes/ComfyUI-Llama/models, restart ComfyUI, hard-refresh the browser with Ctrl+F5, and the nodes live under the LLM menu.

The one genuine trap specific to Advanced: llama-cpp-python doesn't ship official wheels for the Python version ComfyUI uses, so pip may try to build it from source - which means a working C++/CUDA toolchain, and that's where "it won't install" posts come from. If you hit that, grab a prebuilt wheel from a community mirror like JamePeng's llama-cpp-python releases rather than fighting a source build. It's the most common reason people bounce off this pack entirely, and it's fixable in five minutes.