LiteLLMCustomEndpointProvider

The single most common complaint about LLM nodes in ComfyUI is that the model menus go stale. A pack ships with claude-3-haiku-20240307 hard-coded, Anthropic retires it, and suddenly every workflow you load errors out with "model not found." LiteLLMCustomEndpointProvider is the escape hatch: type in any model id, point it at any base URL, and the rest of the pack just works. No waiting for a maintainer to update a dropdown.

This is the node I'd call the most genuinely useful one in the pack. It's also the one the README's RAG pipeline leans on - the pack was tested against Kluster.ai, and this node is why.

What it does

It builds a LITELLM_MODEL config that carries its own api_key, api_base, timeout, and retry settings inline. Downstream completion nodes don't need to know anything about your custom endpoint - they just see a model with credentials baked in. That's a big deal for the provider pattern: you can feed this into LiteLLMCompletionProvider, which then feeds the agent and LightRAG nodes, all against whatever backend you configured.

The inputs

• model_name - the model id at your endpoint, e.g. mistralai/Mistral-Nemo-Instruct-2407.
• api_base - the base URL. Default is https://api.kluster.ai/v1. Point this at http://localhost:11434/v1 for a local Ollama server, or any OpenAI-compatible gateway.
• api_key - your key. Default is literally your-api-key-here, which the rest of the code treats as "no key."
• provider - an enum: openai, anthropic, cohere, or custom. This prefixes the model string (provider/model_name) and tells LiteLLM how to speak to the endpoint.

Optional but worth knowing:

• api_version and organization - for Azure-style or org-scoped endpoints.
• timeout (default 60s) and max_retries (default 3) - set these before blaming the model for slow responses.

Output is a single model (LITELLM_MODEL), ready for any completion node.

Installing

Same as every node here - it's part of ComfyUI_LiteLLM:

cd ComfyUI/custom_nodes
git clone https://github.com/Hopping-Mad-Games/ComfyUI_LiteLLM
cd ComfyUI_LiteLLM
pip install -r requirements.txt

Then restart. ComfyUI Manager users just search "ComfyUI_LiteLLM". No model files, no downloads - this node is pure configuration.

Where people get burned

Three things. First, the api_key you type here lives in the node's widget values, which means it's saved inside your workflow JSON. That's fine for a local machine, embarrassing if you share the workflow. For shared graphs, prefer environment variables (KLUSTER_API_KEY, etc.) and the plain provider nodes instead. Second, "OpenAI-compatible" is doing a lot of work - plenty of proxies claim it and silently drop max_tokens or mangle response_format. If you get weird errors, start with provider set to openai (the most compatible dialect) before trying custom. Third, the default mistralai/Mistral-Nemo-Instruct-2407 model id won't exist on every backend; change it to something your endpoint actually serves, or you'll get a 404 dressed up as a model error. This is the node to reach for when the dropdowns lie - and in this pack, they sometimes do.