LiteLLMCompletionListOfPrompts

Some workflows don't ask the LLM one thing - they ask it fifty things. Prompt variations for A/B testing, batch captioning drafts, a list of questions, one prompt per item in a dataset. Doing that with a single LiteLLMCompletion means fifty nodes or fifty runs. LiteLLMCompletionListOfPrompts takes a LIST of prompts and returns a LIST of completions, and by default it fires them off concurrently. That's the node to reach for when "batch" is the whole point.

How it works

It has two modes, and understanding which one you're in saves you a debugging session:

1. Prompt mode - you feed prompts (a LIST of strings). Each prompt is combined with pre_prompt (if set) and sent as its own completion. If you also feed messages, it expects a list of message histories, one per prompt, so each prompt gets its own conversation context. Length mismatch? It warns and uses empty histories.
2. History mode - if prompts is empty/None but messages is a list of message histories, it iterates the histories instead, using pre_prompt as the new user message for each one. This is the "continue N conversations" mode.

When async is on (default), the calls run through asyncio.gather - concurrent, not sequential, so a batch of twenty prompts doesn't take twenty times as long. If it detects an event loop already running, it falls back to sync rather than crash.

The inputs

• model, max_tokens (250), temperature / top_p (0.5), the two penalties, reasoning_effort - the standard dials.
• pre_prompt (multiline) - prepended to each prompt.
• prompts (LIST) - the batch to process.
• async (BOOLEAN, default true) - concurrency on/off.
• messages (optional, LIST) - per-prompt histories, for either mode.
• use_cached_response (optional).

Outputs: Model, Completions (LIST, one per prompt), Messages (LIST of resulting message sets), Usage.

Installing

ComfyUI Manager, search "ComfyUI_LiteLLM", or:

cd ComfyUI/custom_nodes
git clone https://github.com/Hopping-Mad-Games/ComfyUI_LiteLLM
cd ComfyUI_LiteLLM
pip install -r requirements.txt

Restart, provider key set, done. Under ETK/LLM/LiteLLM.

Where people get burned

The number one surprise is that the pre_prompt default is "Hello World!" - if you leave it alone and your prompts look bizarre, that's why: every prompt in the batch is being prepended with "Hello World!" Change it to empty. Second, concurrency is a rate-limit magnet. Async is great for latency but if you're on a provider with tight RPM limits, twenty simultaneous calls will trip a 429 wall; the underlying completion retries rate limits with a 5-second sleep, so you won't lose the batch, but you'll serialize anyway. For heavy batches, flip async off. Third, watch the length-mismatch trap: messages must line up with prompts one-to-one or you silently lose your context. And since every item is a separate billed call, this node makes runaway costs easy - the Usage output and the pack's caching (use_cached_response) are your friends while iterating on prompts.