OpenAI Compatible LLM

The name is doing the work. "OpenAI Compatible LLM" is a chat-completions client that you point at any endpoint speaking the OpenAI API - OpenAI itself, OpenRouter, Groq, Together, LM Studio, Ollama, vLLM, llama.cpp. One node instead of a pack per provider, and the model list isn't a hardcoded dropdown that rots: it reads GET /models off the endpoint and fills itself.

Why you'd reach for it is the LLM-in-the-graph pattern. You type "a cat," the model hands back a full lighting-lens-composition prompt, and a plain STRING comes out the other side - straight into CLIP Text Encode, Save Text, or any node that takes text. Same node captions a LoRA dataset (every frame of an IMAGE batch is sent as its own image), writes negatives, or merges three text sources into one coherent prompt instead of you gluing strings together. It's a tool bolted onto the graph, not part of the checkpoint - the LLM runs before the sampler, on your GPU or someone's API, and does a text job the diffusion model can't.

How it works

Press Refresh models and the node asks {base_url}/models through a small local route (it does this server-side so your browser's CORS never gets in the way; results are cached 30 seconds, the button bypasses that). On run it POSTs to /chat/completions with your texts as content blocks. Connect a string into text_1 and a text_2 slot appears - autogrow up to 16, empties skipped. Images go in as data: URL blocks, one per frame. Video is sent as a video_url block, which is not part of the OpenAI spec - fine for providers that accept it, a readable error for everyone else.

Two quirks will bite you before anything else. First: seed is never sent to the API. It only decides whether the node re-runs or returns its cached answer. Nothing upstream changed? ComfyUI hands you the old text, every time - set control after generate to randomize on the seed for a fresh result each queue. Second: the model's chat habits are not stripped from the output. There's no decoding constraint or regex cleaner here, so keep system_prompt tight - "reply with the prompt only, no preamble, under 60 words" - or you'll get markdown scaffolding in your conditioning.

The inputs that matter

• base_url - the endpoint root, the part before /models. Hosted: https://api.example.com/v1. Local: http://localhost:11434/v1 for Ollama. Missing https:// and trailing slashes are handled.
• api_key - leave it empty and set OPENAI_COMPATIBLE_API_KEY (or OPENAI_API_KEY) in your environment instead. Type env:MY_VARIABLE to point at a specific one. The reason, spelled out in the tooltip: the key is saved inside the workflow JSON, so a shared workflow leaks it.
• model - filled by Refresh models.
• system_prompt - where the standing instructions live.
• images - optional; needs a vision model.
• reuse_last_result - the sleeper feature. On, the node outputs last time's answer without calling the API at all, stored per-node-per-workflow on disk and surviving restarts. Tune everything downstream without paying for another token.

Output: text (STRING). That's the whole return.

Install

ComfyUI Manager, search "comfyui-openai-compatible" (or just "OpenAI"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/aiko929/comfyui-openai-compatible.git
pip install -r comfyui-openai-compatible/requirements.txt

Windows portable build uses python_embeded\python.exe -m pip install ... instead. The one dependency is aiohttp>=3.9, which ships with ComfyUI anyway - the requirement exists so the package works standalone. Python 3.10+. No model downloads; the model lives on whatever endpoint you point at. It shows up under api/text after restart.

Where people get burned

• "Could not load models" - URL must end in /v1 (or your provider's prefix) and the key has to be valid. The actual HTTP status and body land in the toast and the ComfyUI console.
• 404 on /chat/completions - some providers use a different path prefix; put the full root in base_url.
• Provider rejects temperature - some reasoning models do. Set it to -1 and the field is omitted from the request entirely.
• 400 about image content - that model is text-only. The pack ships tools/probe_modalities.py to ask your endpoint which models actually accept images before you burn a call discovering it.

The honest take: most LLM nodes pick a side in the local-vs-API fight. This one's point is that it doesn't care - base_url to Ollama is free, offline, and uncensored, base_url to a frontier model buys quality, and you flip between them by changing one field. For a young node that's still flying under the radar, that's the right bet.