LiteLLMCompletionProvider

Most nodes in this pack give you an answer. LiteLLMCompletionProvider gives you a function. Wire it up once with your model and sampling settings, and its output - a CALLABLE - is a ready-made "give me a prompt, I'll return a completion" object that every other filter and agent node in the pack can call as many times as it likes.

This is the glue node, and it's quietly the most important one in the agent pipeline. The README's whole RAG story hinges on it: LiteLLMModelProvider → LiteLLMCompletionProvider → DocumentProcessor, where the provider's output becomes the override_callable that drives LightRAG. Same pattern feeds BasicRecursionFilterNode, DocumentChunkRecursionFilterNode, and anything else that asks for a CALLABLE.

What it does

It builds a closure that, whenever called with a prompt, appends that prompt as a user message and runs litellm.completion() with the settings you configured on the node - model, max_tokens, temperature, top_p, penalties. It returns just the text content, not the whole response object. The messages input lets you preload conversation history, so every call the function makes continues from that starting point.

One detail that makes it fit the pack: if the model config it receives is a dict with kwargs (which is what LiteLLMCustomEndpointProvider produces - custom api_key, api_base, retries baked in), the provider extracts those and folds them into every call. So you can point the entire agent/RAG pipeline at a custom endpoint through this one node.

The inputs

• model - a LITELLM_MODEL, from LiteLLMModelProviderAdv or LiteLLMCustomEndpointProvider.
• max_tokens (default 250), temperature / top_p (0.5), frequency_penalty / presence_penalty (0) - the sampling settings baked into every generated call.
• prompt (multiline) - the seed prompt. It's a required input on the node, though for filter use the actual per-call prompts come from the filters, so think of this as a template/example.
• messages (optional) - starting conversation history.

Output: a single Completion function (CALLABLE). Feed it into any LLLM_provider input in the pack.

Installing

It's part of ComfyUI_LiteLLM. ComfyUI Manager, search "ComfyUI_LiteLLM", or:

cd ComfyUI/custom_nodes
git clone https://github.com/Hopping-Mad-Games/ComfyUI_LiteLLM
cd ComfyUI_LiteLLM
pip install -r requirements.txt

Restart, set a provider key in your environment (OPENAI_API_KEY, ANTHROPIC_API_KEY, whatever), and it appears under ETK/LLM/LiteLLM.

Where people get burned

The trap is treating it like a completion node and expecting text out of it. It doesn't return text - it returns a callable, and if nothing downstream consumes a CALLABLE, the node just sits there doing nothing visible. That's by design; connect it to a filter or agent input and it lights up. Second, the max_tokens default of 250 is the pack-wide low-ball - filters that call this repeatedly will clip long completions unless you raise it. And if you point it at a model that wants max_completion_tokens (the o1/gpt-5 "responses" family), the pack's token-override logic kicks in and maps it for you - but double-check your model actually made it into that override list, or you'll see odd token errors. It's a boring node, and that's the compliment: it makes the rest of the pack's machinery possible.