XB-llama - 🚀 模型加载Pro

XB-llama - 🚀 Model Loader Pro is the front door of XB_ToolBox's LLM branch: one node where you pick a local GGUF, cap how much VRAM it may touch, and set every sampling knob the pack's inference nodes will use. If you've been keeping Ollama running just so a ComfyUI node can rewrite your prompt, this is the in-graph alternative.

What it is not is a model loader. Get that straight first.

Why it's called a loader and loads nothing

Read the source: loadmodel() returns two Python dicts. The first (llama_model, type LLAMACPPMODEL) is just a config - model path, mmproj, chat handler, context length, VRAM limit, image-token caps, backend, GPU index. The second (parameters, type LLAMACPPPARAMS) holds the sampling settings. Weights load later, when a downstream inference node fires and calls LLAMA_CPP_STORAGE.load_model().

Two consequences in practice: a workflow containing this node allocates no VRAM just by being queued, and if you swap in an online API node the local weights never load at all. Also, there's no "[XB-llama] loading model" line in the console until the downstream node runs - if you're watching the log wondering why nothing's happening, that's the design, not a hang.

Models come from ComfyUI/models/LLM/ - the pack registers that folder and scans for .gguf, .safetensors, .bin, .ckpt, .pt, .pth. Files with mmproj in the filename are pulled out of the main model list and offered as projectors instead.

The inputs that matter

• model - the enum built from that folder. Empty dropdown means the folder is empty or missing.
• mmproj - None for text-only models. Point it at a projector file and you've built a VLM, with one hard rule the code enforces: chat_handler can't stay None while an mmproj is loaded, or you get an error instead of a model.
• chat_handler - the chat template for your architecture (Qwen3-VL, MiniCPM, Gemma3, GLM-4.6V and friends). This list is discovered from your installed llama-cpp-python, so an old wheel silently gives you a shorter dropdown.- advanced_settings - off by default, and it hides everything below. Flip it on.
• n_ctx (32768) and max_tokens (8192) - the author's tooltips are unusually specific here: a short story fits in 32768, a 56-shot script wants 131072–262144, and 6 shots run about 12K output tokens against ~128K for 56. Under-budget these and the model stops mid-storyboard.
• temperature (0.60) - the tooltip says why: it keeps the [SHOT_START] format strict and cuts hallucination. That marker is the tell for what this branch is built for, which is turning a story into per-shot prompt lines, not writing poetry.
• repeat_penalty (1.05) - a nudge, not a hammer. Too high and the LLM starts dropping dialogue it already wrote earlier in the story.
• vram_limit - the interesting one. -1 means unlimited. Give it a number and the loader estimates per-layer size from the GGUF's file size and layer count, then computes how many layers fit in that budget (subtracting the projector's size when one is loaded). That's how a 9B Q4 lives alongside your diffusion model. The author's own tooltip warns it's a reference figure and you can overshoot - so set it below your free VRAM, not equal to it.
• state_uid - -1 uses the node's own unique id to persist conversation state, so each node keeps its own history.

Skip backend and gpu_device unless you know something I don't. The backend tooltip admits llama-server isn't wired in at all - picking it records the choice and changes nothing - and gpu_device only ever applied to that unimplemented subprocess path.

Outputs, and where they go

llama_model is the config dict that goes into a downstream inference node's model input - the pack's nodes take it as a wildcard type, so the same socket takes either a local config or an online API settings string. parameters carries the sampling settings into that node's optional parameters input. Wire both.

Install

ComfyUI Manager, search XB_ToolBox, or:

cd ComfyUI/custom_nodes
git clone https://github.com/wjluoxiao/XB_ToolBox.git

Restart ComfyUI. Ignore the README's "no extra pip dependencies" line for this branch - requirements.txt lists diskcache and gguf and leaves llama-cpp-python commented out, because you install that wheel yourself: CUDA builds from the JamePeng release page, ROCm/HIP compiled for AMD, matched to your Python version (their example is cp312). Then create ComfyUI/models/LLM/ and drop your GGUFs in it.

Where people get burned

• A RuntimeError about llama-cpp-python not being installed: the wheel isn't there, or it's too old for the chat handler you picked. The error text includes the wheel links.
• Empty model dropdown: empty models/LLM folder, or files in a format outside the extension list.
• mmproj selected, handler left at None: hard error by design.
• Expecting vram_limit to be a promise. It's arithmetic on an estimate; leave headroom.