💬 AITEC LLM Chat

Once an AITEC LLM Loader has a model in memory, AITEC LLM Chat is where the actual conversation happens. It takes the model pin, your system_prompt and prompt, runs inference through llama-cpp-python, and returns the reply as text you can push into a CLIP encoder, a filename builder, or a display node. Multiple Chat nodes can hang off one loader and share the same model - no extra memory per node, which the README is proud of and should be.

The standard wiring is the shortest chain in the pack:

[AITEC LLM Loader] → model → [AITEC LLM Chat] → text

The generation knobs are familiar: temperature (0–2, default 0.7), top_p (default 0.95), max_tokens (default 4096, min 64). The author's tooltip on max_tokens is worth heeding: thinking models want 4096 or more, because their reasoning block eats tokens before you get a single word of answer.

Then come the toggles, and these are the real personality of this node - they exist because Qwen3 and Gemma-class models dump markup into their output:

• remove_think - strips the <think>...</think> reasoning block (and Gemma 4's <|channel|> blocks). On by default, and you'll want it on.
• remove_chatml - when a model repeats <|im_start|>assistant...<|im_end|> blocks, this keeps only the first response. On by default.
• suppress_thinking - appends a "don't think, answer directly" instruction to your system prompt. Off by default, and it's a softer hammer than stripping the tags after the fact.
• reset_kv_cache - on by default. Resets the KV cache before each inference so every run starts clean and doesn't exhaust the context. The tradeoff, per the tooltip: turn it off if you actually want a running conversation history across runs.
• unload_after_run - off by default. On, it frees the model from VRAM after the call (you'll reload it next run). This is your escape hatch for the "LLM + generation OOM" problem.

Outputs are three strings: text (the answer), used_model (which file answered), and status - which is more useful than it looks. A finish_reason=length status means you hit max_tokens; the "context may be exhausted" warning is the pack's way of telling you to raise n_ctx on the loader. If you ever get an empty text with a warning status, that's the fix: more context, not a different prompt.

One subtlety from the source: the node forces re-execution through its cache-bypassing IS_CHANGED hook, so it runs fresh each queue even when your input text is unchanged - useful when you want a new answer without editing the prompt.

Installation is the pack-wide path, plus the manual llama-cpp-python step the README insists on (it's not in requirements.txt - install JamePeng's prebuilt wheel):

cd ComfyUI/custom_nodes
git clone https://github.com/AI-TEC/ComfyUI-AITECCAFE-Toolkit
pip install -r ComfyUI-AITECCAFE-Toolkit/requirements.txt
# then install llama-cpp-python from the JamePeng .whl for your GPU

A practical thought: if you're mainly here to generate prompt text, start with a small quantized GGUF - a 4-bit model fits alongside your checkpoint far more comfortably than a 9B fp16. And when the README's "VRAM overflow" note shows up in practice, unload_after_run is the button.