LiteLLMCompletion

This is the workhorse of the pack. LiteLLMCompletion takes a prompt, calls an LLM, and returns the text - and because it sits on top of LiteLLM, "the LLM" can be OpenAI, Anthropic, Gemini, Bedrock, a local Ollama server, or an OpenAI-compatible endpoint, all through the same node. That's the entire point of the library underneath it: one interface, hundreds of providers, and you swap models by changing one dropdown instead of rebuilding your workflow.

ComfyUI isn't a text tool, which is exactly why this node exists. You're using it inside image or video pipelines - to generate a prompt, summarize an analysis, caption something, or act as the "brain" that decides parameters before a KSampler runs. The pattern that makes sense in practice: feed it a prompt built from your workflow state, and wire its Completion output into a text node or back into the graph. It's an API call, so your machine's GPU is irrelevant; what matters is your key, your prompt, and your budget.

How it works

The node appends your prompt as a user message, then calls litellm.completion() with the model and sampling parameters. A few details worth knowing:

• Zeroed sampling params (temperature=0, top_p=0, penalties of 0) get dropped rather than sent - some providers reject explicit zeros, so it quietly normalizes for you.
• litellm.drop_params = True means unsupported parameters are discarded instead of erroring. That's why a reasoning_effort of "low/medium/high" won't kill the call on a model that doesn't know what reasoning effort is.
• Rate limits are retried with a 5-second sleep in a loop. Costs can still spike - see below.
• Every response is cached to a JSON file in ComfyUI's data folder, which is how use_cached_response works when you turn it on.

The inputs that matter

• model - connect LiteLLMModelProviderAdv or LiteLLMCustomEndpointProvider here. The default anthropic/claude-3-haiku-20240307 is ancient; don't rely on it.
• prompt - multiline, this is your instruction. This is a chat-model prompt, so write it like an instruction, not a tag soup.
• max_tokens (default 250), temperature and top_p (default 0.5) - the sampling dials you actually tune. 250 max tokens is tight for any real output; bump it.
• reasoning_effort - low/medium/high, passed through for models that support it.
• messages (optional) - a previous conversation to continue.
• use_cached_response (optional) - replay a cached answer for the same inputs. Handy in development, a footgun in production.

Outputs: Model, Messages (the conversation, now including the reply - chain it into the next call), Completion (the text), and Usage (token counts, plus a green "cached response used" or red "new result generated" marker).

Installing

The pack installs via ComfyUI Manager (search "ComfyUI_LiteLLM") or:

cd ComfyUI/custom_nodes
git clone https://github.com/Hopping-Mad-Games/ComfyUI_LiteLLM
cd ComfyUI_LiteLLM
pip install -r requirements.txt

Then restart, set your API key as an environment variable (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.) and you're live. The requirements.txt pulls in a lot of stuff you don't need for this one node - LightRAG, sentence-transformers, neo4j - but those are for the RAG half of the pack; if one of them fails to import, the core LLM nodes still load.

Where people get burned

The stale default model is the classic trap - if you hit a "model not found" error on a fresh install, it's usually that. Fix it with a current model id. Second, remember this node's retry loop only handles rate limits; a 429 with a long cooldown, or a 401 from a wrong key, will just surface as an error. Third, LLM calls are money. Wire use_cached_response on during iteration and turn it off for production. And the usual security note for any LLM custom node: your prompts leave your machine for a third-party API, and custom nodes run arbitrary code - keep ComfyUI off the public internet and only install packs you'd actually vouch for.