Groq Batch

Some jobs aren't one prompt, they're fifty. Translating a list of captions, rewriting a folder's worth of prompts, classifying a pile of short texts - chaining fifty Groq Chat nodes is madness, and running them one at a time is a waste of an API that's famous for being fast. Groq Batch is the middle path: paste in a JSON array of prompts, set one model, and it fires them all concurrently with a cap on how many hit the API at once.

The payoff is simple - one node, one execution, a batch done. And because it collects per-item results, one bad prompt doesn't take the whole run down with it.

How it works

Under the hood it's asyncio driving Groq's async client. Each prompt becomes a task, a semaphore enforces your max_concurrent limit, and the tasks run in parallel until every prompt has either produced a response or failed. Failures are caught individually and bucketed into an errors list instead of aborting the batch. That separation - successes and errors as separate outputs - is the feature that makes this node usable unattended.

One honest clarification while we're here: despite the name, this is not Groq's offline Batch API (the one for fire-and-forget bulk jobs that complete later). This is live parallel chat requests, so you get answers in the same execution. If you want true offline bulk processing, Groq's own batch endpoint is a different thing entirely.

The inputs that matter

• prompts_json - a JSON array of strings, one per job. The default shows the exact format:

  [
    "Tell me a joke",
    "What is 2+2?",
    "Name a color"
  ]

  Every element must be a string; the node validates that before it starts.
• model - one model for the whole batch, drawn from the same chat list as Groq Chat (llama-3.3-70b-versatile default, llama-3.1-8b-instant for speed, the gpt-oss pair, the groq/compound agentic models, llama-guard-4-12b).
• temperature and max_tokens - applied to every prompt in the batch.
• system_prompt - optional; if set, prepended to every message, which is how you'd do "translate all of these to French."
• max_concurrent (1–20, default 5) - the throttle. This is the knob you'll actually fight with.

The three outputs are responses_json (each prompt with its index, response, and token usage), summary (a one-line tally: total / success / failed, plus total tokens), and errors_json (anything that failed, with the error text).

Installing it

Identical to the rest of the pack. ComfyUI Manager → search "Groq Nodes", or:

cd ComfyUI/custom_nodes
git clone https://github.com/brayevalerien/ComfyUI-GroqNodes
cd ComfyUI-GroqNodes
pip install -r requirements.txt

Restart, set your GROQ_API_KEY, and you're running. Light deps, no model downloads - the pack is a pure API client.

Common issues

The classic beginner trap is max_concurrent plus the free tier's rate limits: crank it to 20 and you'll start seeing 429 errors land in errors_json even though the node retries with backoff. Keep it around the default 5 unless you're on a paid key, then ratchet up and watch errors_json. Also mind the JSON - it must be an array of strings, and a formatting slip shows up as "Invalid JSON in prompts" / "All prompts must be strings". Since failures surface in errors_json rather than crashing, skim that output before trusting the batch. Standard cloud-API caveats apply - your text leaves the machine, and this is a young one-maintainer pack, so install from the real repo and keep it updated.