OpenAI API - Batched Chat Completion

Why this exists

Say you have eight reference images and you want a caption for each one. Or four storyboard panels that each need a different prompt. With a single Chat Completion node you'd wire eight of them, or run the workflow eight times and save the answers by hand. Nobody wants that.

Batched Chat Completion does the boring thing well: it takes a list of prompts and/or a list of image groups, sends one independent request per item, and returns the responses in input order. Position one goes to position one, always. That ordering guarantee is the whole product, because downstream everything depends on the first response belonging to the first image.

It's the node that turns "an LLM in my graph" into "a dataset tool in my graph" - sixty images through a vision model in one queue, captions landing beside the files. That's the captioning job people actually do, and it's the same job a local VLM like JoyCaption does offline for free. The API path earns its place when you want a stronger model, don't have the VRAM, or are captioning a set small enough that a few dollars is cheaper than two hours of your evening.

How it works

This node is declared as an input-list node, which is ComfyUI's mechanism for "give me the whole list, I'll handle iteration myself." So prompts arrives as the entire STRING list from upstream - not one item at a time - and the node walks it in a for loop, calling the same completion routine the single node uses, once per index, checking for cancellation between requests so you can hit interrupt on request 40 of 60.

Shared inputs are checked for being genuinely shared: client, model, system_prompt, options and force_regen must each resolve to exactly one value, because there's one client and one model for the entire batch. image_batches is likewise a single payload - a grouping of lists-of-images, not a flat list of images - which is why the helper nodes matter so much here. If prompts and image_batches are both supplied they must be the same length; the node compares them and refuses with a message naming both counts.

Each request is independent. There's no history carried between invocations, by design: "load this skeleton, now this colour, now this light" is a conversation, not a batch. Want a conversation? Use the single-node version and its History output.

The requests are sequential, not parallel. Twenty images means twenty round trips, one after another, each subject to the Client node's timeout. Slow and predictable, which is the right trade for a node whose output order you're relying on.

Inputs and outputs

• API Client (client) - required, from the Client node. Exactly one.
• Model (model) - required, one shared model for every request.
• Force Regen (force_regen) - default false; true regenerates every response every run.
• System Prompt (system_prompt) - one shared system prompt for all requests. This is where you put your output contract ("respond with only a Danbooru-style tag list").
• User Prompts (prompts) - a STRING execution list; feeds from Multiline String to List, Repeat String to List, or the Responses output of another batch node.
• Image Batches (image_batches) - from Image Batch List or Image Batch to List. Each group goes to the request at the same position. Use a vision-language model.
• Options (options) - one shared set, from any of the option nodes.

One output: Responses, a STRING execution list in input order. Wire it into CLIP Text Encode List if you're generating, or Multiline String to List if each response is multiline and you want each line separated out.

Install

ComfyUI Manager → search OpenAI API or Comfpod, or:

cd ComfyUI/custom_nodes
git clone https://github.com/lany42/comfyui-openai-api
cd comfyui-openai-api
/path/to/ComfyUI/.venv/bin/python -m pip install -r requirements.txt
# restart ComfyUI

Dependencies are the openai SDK v2, numpy and Pillow. No weights to download. The pack needs Python 3.13+.

Where people get burned

"prompts and image_batches must have equal lengths." The classic version of this: you split an 8-image batch into 8 groups and connect a text box with one prompt. Connect the Count output from Image Batch to List into Repeat String to List's Count instead, and you get 8 identical prompts - right shape, no thinking.

"must contain exactly one shared value." You connected something that emits a list of two clients or two models. Rare, but it happens when you copy-paste and forget to prune.

No images get sent. If image_batches is unconnected, every image group is silently absent and you're asking a text model to caption nothing. Check the wiring before blaming the model.

Empty strings in the output. A refusal, a rate limit that exhausted its retries, or an image the model won't discuss comes back as "". Order is preserved but the content isn't - always eyeball the Responses before you feed them into a CLIP encoder, where an empty string quietly becomes empty conditioning.

Cost. This node multiplies your bill by the batch size, images included. Ten images × a big vision model at full resolution is a real number. Test on two before you queue fifty.