Griptape Tool: Audio Transcription

This is the node that turns a Griptape agent from a text-only assistant into one that can listen. It's an agent tool: you hand it to an agent, and the agent decides to use it when a prompt involves audio - a voice memo, a podcast clip, a recording you dropped into the workflow. The tool transcribes the audio to text, and the agent can then work with that text however you've directed it. If you're building a "upload an interview, get a summary" or "narrate instructions, agent acts on them" pipeline, this is the piece that closes the loop.

One honest caveat up front: the transcription itself isn't magic under the hood. If you don't wire in your own driver, the node defaults to OpenAI's whisper-1, which means you need an OPENAI_API_KEY set in the pack's Settings → Griptape section (or your environment) or nothing will come back. That's the same key setup as every other OpenAI-backed piece in this pack, so if you've already got the agents running, you're done.

How it works

The node builds an AudioTranscriptionTool and hands it to your agent as a TOOL_LIST. When the agent judges that audio needs transcribing, it invokes the tool, which passes the audio to the transcription driver and returns text. You can stay on the default Whisper driver, or plug in your own driver socket (an AUDIO_TRANSCRIPTION_DRIVER from the pack's driver nodes) if you want something faster, cheaper, or hosted elsewhere.

The inputs that matter

• off_prompt - default True. This one matters more than it looks. On, the transcript stays out of the agent's working context (the tool runs off-prompt and only the result you ask for is surfaced); off, the raw transcript goes into the model's context window. For sensitive audio, keep it on - it's also cheaper, since you're not feeding long transcripts to the LLM as tokens.
• driver - optional. The transcription driver. Defaults to OpenAI whisper-1.

One output, TOOL (TOOL_LIST), which plugs into an agent's tools input - either on a Create Agent node or by replacing tools on an existing agent.

Install

Same as every node in this pack:

cd ComfyUI/custom_nodes
git clone https://github.com/griptape-ai/ComfyUI-Griptape

or ComfyUI Manager → search "Griptape" → install. Restart ComfyUI and the node appears under the right-click Griptape menu, Griptape/Agent Tools.

Where people get burned

The top gotcha is forgetting the key - default driver is OpenAI, no key, no transcript, and the failure can be quiet. Set OPENAI_API_KEY in Settings → Griptape before you wire anything up. Second, remember the pack's heavyweight dependency: griptape[all] drags in its own torch, and on Nvidia boxes that can replace the CUDA build ComfyUI expects. If your whole graph starts erroring after installing this pack, the README's fix is:

pip uninstall torch
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu121

And if the audio input to your workflow is a file the agent can't find, that's not this node - check the FileManager tool for granting the agent disk access.