Local LLM Server

Local LLM Server exists for one specific headache: a chat model and an image model do not fit on the same GPU at once. If you've ever pointed a client like SillyTavern straight at llama-server while also generating images in ComfyUI, you know the failure - ComfyUI (or you) kills the LLM process to free the card, and the next chat message hits a closed port. Nothing is listening, so nothing can start the model back up. SillyTavern is still a genuinely live ecosystem in 2026, and the whole point of a local setup is keeping it off one card with your diffusion.

The node's answer is to split the listener from the model. It launches a real llama-server (or koboldcpp) as a subprocess under ComfyUI, behind a small gateway HTTP server that this ComfyUI process owns:

• the gateway starts when you queue this node once and stays up as long as ComfyUI runs. That's the address you configure in the chat client and never touch again;
• the model is a separate process on a private port that holds the VRAM. It loads on the first request that needs it and is killed whenever the VRAM is wanted elsewhere.

A chat message arriving with the model unloaded just waits for the load, then gets forwarded - the client sees a slow first reply, never an error. And requests that need no model, like the model list and health probe behind SillyTavern's Connect button, are answered by the gateway itself, so connecting works with nothing loaded at all.

The inputs that matter

It's a config-heavy node, but a beginner really sets five things:

• action - serve (start the gateway, wait for the first request), serve + load the model now, reload the model, or stop everything.
• server_binary - an executable to launch. This is the big gotcha: nothing is bundled. You download llama-server yourself from a llama.cpp release (or grab koboldcpp) and drop it under ComfyUI/models/llm, subfolders fine - the dropdown auto-lists anything it finds there recursively. If it's elsewhere, pick the placeholder and type a path in server_binary_path.
• model - the .gguf from ComfyUI/models/llm to serve (or type a full path in model_path).
• n_ctx (default 8192) and n_gpu_layers (-1 = all) - the usual two, and context is what eats VRAM fast once a long chat plus a character card fills it.
• listen_port (default 5001) and listen_host (127.0.0.1 = this machine; 0.0.0.0 to reach it from the LAN).

Everything else is well-labelled and mostly "leave it alone until it bites": extra_args appends raw server flags (--flash-attn, --jinja, --model-draft …) and changing it reloads the model; autoload and unload_comfy_models handle the VRAM hand-off; fix_params repairs request bodies llama.cpp refuses (SillyTavern famously sends dry_sequence_breakers and friends as JSON strings where an array is required), and auto_retry drops an unknown field and retries when a 400 names one. free_on_prompt is the safety net: any queued prompt that isn't itself about the server unloads the model as it's queued, before a node of it runs. idle_unload_minutes frees VRAM after a quiet spell.

Four outputs: status, base_url (ends in /v1 - paste it into SillyTavern as a Custom OpenAI-compatible endpoint with no API key), server_log (your debugging friend when a load stalls), and help (a cheat sheet - wire it to a Preview as Text node). The gateway also answers /kinburg/status, /kinburg/unload and /kinburg/load in a browser.

Installing

Clone the pack or install "Kinburg-Nodes" via ComfyUI Manager and restart. One clarification people trip on: the pack's install.py installs the CUDA build of llama-cpp-python, but that's for the in-graph Local LLM (GGUF) nodes - this node doesn't use it at all, because it shells out to the external server binary you provide. If you only want the server pair, the binary and a .gguf in ComfyUI/models/llm are the actual prerequisites.

Where it bites

Model loads are slow the first time - raise startup_timeout (default 300s) if a big quant legitimately needs longer, and read server_log rather than guessing. Changing the model, n_ctx or extra_args reloads on the next request. And remember this node generates nothing itself: if you want a text answer inside a graph, that's Local LLM (GGUF), not this. This one is the sidecar - the thing that makes a local chat client and your image pipeline share one card without ever seeing each other's mess.