📦 AITEC LLM Loader

The whole "LLM inside ComfyUI" story in this pack starts here. AITEC LLM Loader takes a local GGUF (or safetensors) model and hands it to other nodes over a MODEL pin - and because the loaded model is cached by filename + settings, ten Chat nodes downstream share one in-memory copy instead of loading ten. That sharing is the point: it's what lets you drop a local model into a workflow without blowing up VRAM ten times over.

Under the hood it's llama-cpp-python's Llama class. The loader scans ComfyUI/models/llm/ for *.gguf, *.GGUF, and *.safetensors files and lists them in the dropdown - if you see "(no models found)", that's the model folder telling you it's empty. Models go in:

# put your .gguf files here
ComfyUI/models/llm/

The dropdown also lists safetensors files, but the README (and most community practice with this stack) recommends GGUF when you're running it alongside image generation - quantized, smaller, and friendlier to shared VRAM. The author points at HauhauCS' uncensored Qwen/Gemma GGUF builds as examples.

The three inputs are the whole node:

• model_file - which model from the folder (the enum dropdown).
• n_ctx - context window, default 4096, up to 131072. The author's tooltip pushes Qwen3-style thinking models to 16384+. This matters more than you think: reasoning models burn context on their <think> blocks, and an exhausted context is the classic "model returns empty string" failure in this pack.
• n_gpu_layers - layers offloaded to the GPU; -1 (the default) sends everything to the GPU.

Output is a single model of type LLM_MODEL, which only the pack's own Chat/Vision nodes understand. This isn't a standard ComfyUI type - don't try to wire it into anything else, because nothing else knows what to do with it.

Now the install gotcha, and it's a real one: llama-cpp-python is NOT in the pack's requirements.txt. The README explicitly tells you to install it yourself from a prebuilt .whl, and links to JamePeng's release page. If you only install the pack via Manager, the four LLM nodes will sit there throwing "llama_cpp is not available" until you install it. On Windows especially, don't try pip install llama-cpp-python and hope for a compile - the prebuilt wheel is the path of least resistance:

cd ComfyUI/custom_nodes
git clone https://github.com/AI-TEC/ComfyUI-AITECCAFE-Toolkit
pip install -r ComfyUI-AITECCAFE-Toolkit/requirements.txt
# then, separately, the LLM piece:
pip install <the JamePeng llama-cpp-python .whl matching your CUDA/ROCm>

Manager installs the rest of the pack (which drags in tensorflow and opennsfw2 for the moderation nodes - heavy but one-time). One thing to know before you run a text-to-image workflow with a local LLM loaded: the README's VRAM warning is accurate. Generation models plus a loaded LLM coexist only if your card has room for both; loading a 9B model next to a checkpoint can OOM immediately. When that happens, either shrink the model (n_gpu_layers offload, a smaller quant) or use unload_after_run on the inference node. Load the model, run the LLM, let it go, then generate.