Griptape Prompt Driver: Groq

Groq is the ridiculous-speed LLM API: they run open models on custom hardware that makes Llama and Mixtral respond fast - thousands of tokens a second, to the point that it feels broken at first. For a Griptape agent, that speed matters more than you'd think, because agent workflows make many round-trips: the model thinks, calls a tool, waits, thinks again. Each step at Groq speed is nearly instant, so a multi-step agent feels interactive instead of glacial. There's also a generous free tier, which makes this the default answer to "I want a hosted brain and I'm cheap."

The catch: you're limited to what Groq serves, and it's open models only - no GPT or Claude here. If you want Llama 3, Gemma 2, or Mixtral at absurd speed, this is your driver.

The inputs that matter

• model - the biggest dropdown in the pack, 12 options. Default gemma2-9b-it; the interesting ones for agents are llama-3.3-70b-versatile (the smartest all-rounder) and llama3-70b-8192 (older but dependable). There are vision-preview Llama 3.2 models if you need image input, and llama-guard-3-8b if you're building moderation.
• api_key_env_var - env-var name, default GROQ_API_KEY. Free key at console.groq.com.
• min_p (default 0.1) - Groq serves models with strong tokenizers, so min_p is the sampling knob that matters most here; lower it for more creative output.
• response_format - default or json_object, for machine-parseable agent output.
• max_tokens - -1 default.

Standard temperature, seed, use_native_tools, max_attempts_on_fail included. Output: DRIVER.

Why this over a local model?

People who try Groq often bounce between it and Ollama. Groq wins on quality-of-life: no VRAM, no model downloads, no GPU contention with your image generation. Ollama wins on privacy and zero ongoing cost. If ComfyUI is already eating your VRAM for diffusion, Groq is the saner choice - the model runs on their hardware, not yours.

Installing it

Pack install, standard: ComfyUI Manager → Griptape → ComfyUI-Griptape, restart, or

cd ComfyUI/custom_nodes
git clone https://github.com/griptape-ai/ComfyUI-Griptape

Set GROQ_API_KEY in Settings → Griptape or your environment.

Common issues

Groq models come and go, and their dropdown options can go stale - a deprecated model ID returns an error even with a valid key; pick a current one. Rate limits on the free tier are real and will surface as 429 errors on a chatty agent; Groq is fast, but the free tier throttles fast too. And the pack-wide torch/griptape-version issues apply as usual: reinstall torch with cu121 wheels if ComfyUI breaks, and python -m pip install griptape -U if you see import errors.