Llama Server Chat/Vision (Tlant)

The name is a hint: Llama Server Chat/Vision (Tlant) isn't a model - it's a client for one. It talks to any OpenAI-compatible chat server you already run (a llama.cpp server, Ollama, vLLM, LM Studio), sends a prompt, and drops the answer back into your graph as text. And unlike a lot of "LLM node" packs, it costs you nothing beyond what you're already running, because there's no API key required for a local endpoint - the key field just sits there unused until you point it at a paid server.

This is the node that makes the rest of the Tlant toolkit actually do something. The MiniMax H3 prompt nodes in this pack (see the MiniMax H3 articles) output an instruction for a remote stateless LLM, not the final prompt. LlamaServerChat is the natural bridge: feed it that instruction and it returns the clean English MiniMax H3 prompt the video workflow expects. It also does plain captioning and prompt-rewriting duty - the default prompt is Chinese for describing an image, but you'll swap that for whatever your job needs.

Vision works out of the box. Connect an IMAGE to the optional images input and the node base64-encodes it (PNG or JPEG, controllable via image_format and jpeg_quality) and embeds it as a data URL in the chat message, up to max_images of them. No separate vision server configuration to fuss with - if your endpoint handles image content, this just works.

The inputs that matter most:

• server_url + endpoint - defaults http://127.0.0.1:18080 and /v1/chat/completions. Point them at your server; the endpoint can be a full URL if you need to bypass the base.
• model - the model name your server knows. Many llama.cpp setups accept default.
• system_prompt, prompt - the instruction and the user message.
• max_tokens, temperature, top_p, top_k, min_p, repeat_penalty, plus the usual penalties - a genuinely complete sampler panel for tuning local models.
• response_format - text or json_object; the JSON mode makes the node usable as a structured-data step.
• extra_json - raw JSON merged into the request payload, the escape hatch for server-specific options.
• unload_after - after the call, POSTs to /models/unload to free VRAM. Nice on a single-card machine where the LLM and the diffusion model share memory.

Three outputs: text (the assistant's message, cleanly extracted), raw_json (the full server response, for when you need the details), and status (a short string with HTTP code, elapsed time, image count, and unload info).

Install

ComfyUI Manager → search ComfyUI-Tlant-Toolkit → install, or:

cd ComfyUI/custom_nodes
git clone https://github.com/Tlant/ComfyUI-Tlant-Toolkit

Restart ComfyUI. The pack has zero third-party dependencies and downloads nothing.

Troubleshooting

• Connection refused / timeout. The default port is 18080 - that's not llama.cpp's stock 8080. Either start your server on 18080 or change server_url. Test the endpoint in a browser first; if curl can reach it, the node can too.
• Non-JSON or HTML response. If the server returns a 404 page, you've got the wrong endpoint or base URL. raw_json/status will show you the actual HTTP code.
• It always runs. Like the other nodes here, it reports itself dirty every execution, so every queue run makes a fresh API call - no cache to accidentally replay an old answer. That's fine, but keep timeout_seconds sane for long generations.