Local LLM

Here's the pack's irony: everything else in ComfyUI-API-DockerCPU exists to offload generation to cloud APIs, and then this node shows up and points the other way - at a local LLM server. The default endpoint is http://localhost:1234, which is LM Studio's default port, and the whole idea is that a CPU-only box can still run a small chat model locally (LLMs are far friendlier to CPU-only than diffusion models). Give it a prompt, it asks the local model, and returns the response as a string you can feed back into your workflow - rewriting prompts, captioning, naming files, driving decisions. It's the pack's "thinking" node.

Two API styles are supported via api_type:

• openai_compatible - the default. It posts to <endpoint>/v1/chat/completions with a standard OpenAI-shaped payload, so it works against LM Studio, Ollama's OpenAI-compatible mode, llama.cpp server, vLLM, or any hosted OpenAI-compatible API. Point api_key at whatever that server wants (usually blank for local).
• lmstudio_native - LM Studio's native API, which adds a couple of tricks: unload_existing_models (default false) unloads whatever's loaded so your model can fit, and the node can load a specific model instance itself.

Inputs that matter:

• endpoint_url (default http://localhost:1234), api_key (blank by default - most local servers don't want one).
• model - the only choice in the current schema is auto, which means the node queries the server's /models endpoint and picks the first available model. If your server has several models loaded, auto grabs the first, and there's no dropdown to pick another - a limitation worth knowing.
• prompt and system_prompt - both multiline. The system prompt is your "you are a prompt engineer" framing; the prompt is the actual question.
• temperature (0.7), max_tokens (1024), timeout (120s) - the usual dials. On a CPU-only box, small models can be slow; if you're getting timeouts, raise timeout.
• image (optional, IMAGE) - vision-capable models can take an image, which turns this into a local captioning node. The image gets converted to a data URI and included in the payload. Nice complement to the cloud VLM node in the Replicate section.
• force_rerun - the pack-wide cache buster, useful here because ComfyUI will otherwise cache the node's output and you'll get the same answer forever.

Outputs: text (the model's reply) and model (the model ID that actually answered - handy when auto picked and you want to know what it chose).

Install is the pack standard - ComfyUI Manager (search ComfyUI-API-DockerCPU), or:

cd ComfyUI/custom_nodes
git clone https://github.com/trustypangolin/ComfyUI-API-DockerCPU
cd ComfyUI-API-DockerCPU
pip install -r requirements.txt

No token required (it's local); restart and find it under 🎨 DockerCPU API/Utilities.

Real-world notes: the "auto" model picker is the node's weak spot - if your server reports models in an order you don't like, you can't choose, so it's worth having one model loaded when you run it. And unlike the cloud nodes, there's no dry_run here, so a typo'd endpoint just fails with a connection error (the error message includes the server's response body, which helps). If you're running this in the pack's intended CPU-only Docker setup, keep the model small - a 3-8B GGUF is the sweet spot for prompt-rewriting and captioning on CPU, and it keeps the whole workflow local and free.