XB-llama - 💬 指令推理

This is the workhorse of the XB-llama stack - the node that actually talks to your local model. XB_llamaInstruct takes a loaded LLAMACPPMODEL, runs a chat completion with whatever prompt you set up, and hands back the text - plus a per-line list and a state ID you can use to keep the conversation going. It's the node behind almost every "ask a local LLM something inside ComfyUI" workflow, and it handles plain text and vision, since it can take images and, in video mode, treat a frame sequence as a clip.

The prompt system

You get three ways to steer it:

• preset_prompt - 23 built-in presets. The interesting ones for image work: the "Prompt Style" family (tags / simple / detailed / cinematic), "Creative - Refine & Expand Prompt," and - the star - "Vision - *Bounding Box [EN]", which asks the model to locate objects and report {"bbox_2d": [x1, y1, x2, y2], "label": ...} JSON. That's the output you feed straight into XB_llamaJSON2BBox and then on to a mask or SEGS.
• custom_prompt - your own text. If the chosen preset contains a # placeholder (the Bounding Box one does), your text replaces it - so you type the object category you want located. Otherwise it overrides the preset entirely.
• system_prompt - optional system framing. Wire in the output of the preset-builder nodes (XB_llamaPromptEnhancer, the MiniMax presets, XB_llamaStoryboardEnhancer) here and you get a fully assembled system prompt for free.

Vision and video handling

• inference_mode - one by one reads images one at a time (each gets its own completion, results listed per image), images batches them into a single completion, video treats the input frames as a video clip.
• max_frames - only for video mode: how many frames to uniformly sample from the input.
• max_size - downscale cap (default 256) for images/video before sending, which keeps token cost and VRAM sane.

Feeding images requires the loaded model to have a vision handler (chat_handler) and, for LLaVA-family models, a loaded mmproj - the node raises a clear error if you try images on a text-only model, so it won't silently garbage out.

The other inputs and outputs

• parameters (optional) - a LLAMACPPARAMS wire from XB_llamaParameters. Leave unplugged and it uses sensible built-in defaults.
• seed - reproducibility for the completion.
• force_offload - unloads the model after inference to free VRAM for the diffusion side (the workflow this pack is really about: LLM first, then the big video model needs all the VRAM).
• save_states - keeps this conversation's history in memory under a state ID, so a later run can continue the thread.
• queue_handler - an any-type wire you can use to control execution order of instruct nodes in a chain.

Outputs: output (the full text), output_list (the text split into lines - handy for feeding XB_ListDispatcher or storyboard processors), and state_uid (the conversation state ID, for XB_llamaCleanStates).

Installing it

The pack install is standard (ComfyUI Manager → XB_ToolBox, or git clone from the repo), but this node hard-requires llama-cpp-python - see the XB_llamaModelLoader article for the CUDA/ROCm install, because that's where the "no extra dependencies" claim breaks down. Models go in ComfyUI/models/LLM/.

Where people get burned

The biggest gotchas: (1) forgetting the mmproj + chat_handler on vision - the error message is in Chinese and easy to skim past; (2) save_states left on across runs makes the model "remember" old context and answer differently than you expect - turn it off unless you actually want multi-turn continuity, and use XB_llamaCleanStates to reset; (3) local 8B models hallucinate - for BBox detection, verify boxes before trusting them downstream. The one by one mode also generates per-image completions, which can be slow; images mode is the speed pick when a single answer over all frames is fine.