ComfyUI-LLMClient

ComfyUI-LLMClient

LLM nodes for ComfyUI over any OpenAI-compatible API (llama.cpp, vLLM, NInfer, Strata, OpenAI, ...): prompt generation and enhancement, image captioning and analysis, text extraction — output routes to any string socket in your graph.

No new dependencies: aiohttp, Pillow, and torch already ship with ComfyUI.

Install

• ComfyUI-Manager: search ComfyUI-LLMClient, or install via git URL https://github.com/Vex131/ComfyUI-LLMClient
• comfy-cli: comfy node install comfyui-llmclient
• Manual: clone into ComfyUI/custom_nodes/ and restart

Requires a running OpenAI-compatible server, e.g.:

llama-server -m Qwen3-8B-Q4_K_M.gguf --port 8080

Quick start

1. LLM Provider: set base_url (http://127.0.0.1:8080/v1; /v1 is appended if missing), click Test Connection, pick a model.
2. LLM Chat: connect the provider's model output, write a prompt.
3. Wire output wherever you need it — CLIP Text Encode, Save Text, etc.

Cancel stops generation on the server too, not just in ComfyUI.

Nodes

LLM Provider

• base_url — OpenAI-compatible API root; /v1 appended if missing.
• api_key — optional; stored in the workflow JSON (see warning).
• model — combo filled by Test Connection. The probe auto-detects the server profile (llama.cpp / vLLM / generic) and caches models, context size, and vision flags in providers.json; also filled on first execution.
• Outputs: model handle for LLM Chat / LLM Unload, and info text (profile, context, unload support, per-model vision flags).

LLM Chat

Streams the reply.

• prompt, optional system_prompt, optional image.
• vision — auto rejects images for models detected as text-only; yes forces it.
• model_config — optional sampling overrides (see LLM Model Config).
• seed — -1 = random.
• Outputs: output, and reasoning (thinking trace when the model emits one).

LLM Model Config

Optional sampling overrides. -1 = provider default for temperature, top_p, top_k, max_tokens. reasoning_effort is free text, empty = provider default; use only levels your server accepts (low, medium, high, xhigh, ...) — invalid values are rejected by the server.

LLM Unload

offload_vram_to_ram / release_all / wake_up. Support depends on the server (matrix below); unsupported servers fail with a clear error.

Connect LLM Chat's output to the optional text input, then use LLM Unload's text output downstream to ensure generation finishes before unloading and downstream processing waits for the unload command. Text is passed through unchanged; status remains the first output. Leave text unconnected for standalone commands such as wake_up.

Unload support

| Server | What works | Requirements | |--------|-----------|--------------| | vLLM | sleep level 1 (weights to RAM), level 2 (discard), wake | --enable-sleep-mode and VLLM_SERVER_DEV_MODE=1 | | llama.cpp | /models/load, /models/unload | router mode (--models-dir) |

API key warning

The API key is a workflow input: when set, it is stored in the workflow JSON. Anyone you share the workflow with can read it.

Roadmap

Planned: agentic loops and RAG, document/text input, tool use.