Ollama Audio Transcribe

Ollama Audio Transcribe turns speech into text without you leaving the graph or paying per minute. Feed it any AUDIO output - a LoadAudio node, a clip from a video workflow - and it returns a plain STRING of what was said, using a Whisper model that Ollama itself serves. It's the audio version of the captioning job the LLM-in-ComfyUI crowd already does for images, and it's free, offline, and uncensored in the sense that matters: no audio ever leaves your machine.

Why bother when there are a dozen transcription tools? Because in ComfyUI the output is wired, not pasted. The transcript drops straight into a text node, which can feed a prompt enhancer, get embedded for retrieval, or drive an LLM that summarizes a meeting recording while your diffusion runs. You're composing tools instead of copying text between apps.

How it works

Ollama has no native audio API, so this node uses the same trick as the rest of the pack's audio nodes: the audio is converted to 16kHz mono 16-bit WAV with a full RIFF header, base64-encoded, and passed through Ollama's images[] field. Ollama detects the RIFF/WAVE magic bytes and routes it to the model's audio encoder. The resampling and mono mix happen automatically, so whatever sample rate your audio node emits, the node normalizes it first.

It calls client.generate() with a prompt that defaults to "Transcribe the audio exactly as spoken." - plus, if you set one, a language hint appended as "The spoken language is X." and a temperature for sampling.

The inputs that matter

• audio - required. Connect from LoadAudio or any AUDIO output.
• url - your Ollama server, defaulting to http://127.0.0.1:11434.
• model - the speech-to-text model. The dropdown pulls from your installed Ollama models, so pull one first (below).
• prompt (optional) - a hint that guides transcription. This is where you add vocabulary the model won't know - names, jargon, product terms - or describe speaker style. Whisper will happily misspell your username unless you tell it.
• language (optional) - a BCP-47 code like en, fr, or ja. Leave it blank for auto-detect; if the transcript comes back gibberish in the wrong language, pin it.
• temperature - 0 by default, which is what you want for transcription: deterministic. Only nudge it up if you're getting weird repetitive output.

Output: text, a single STRING of the transcript. Wire it into a text display node, an LLM prompt, or wherever the words need to go.

Get a Whisper model first

The node is a thin shell; the brains are in Ollama. Pull a Whisper model or it won't have anything to transcribe with:

ollama pull whisper-large-v3-turbo

Smaller Whisper variants exist if you're short on VRAM - and note this is Whisper running inside Ollama on your GPU, which is exactly the whisper+ollama local combo people have been wiring by hand for years, minus the glue code.

Install

Part of the ComfyUI-OllamaOmni pack. ComfyUI Manager → search "OllamaOmni", or:

cd ComfyUI/custom_nodes
git clone https://github.com/ckinpdx/ComfyUI-OllamaOmni
pip install -r ComfyUI-OllamaOmni/requirements.txt

Then restart ComfyUI. The dependency list is just ollama, so nothing heavy sneaks in.

Troubleshooting

• Empty or silent transcript - your audio didn't reach the model. Check the audio actually has signal, and that you're not feeding raw PCM (the node needs a proper AUDIO node output; the RIFF-header requirement is handled internally).
• Wrong language output - set language explicitly instead of trusting auto-detect.
• Model dropdown is empty - Ollama isn't running, or ComfyUI restarted before you pulled the model. Start the server, ollama pull, restart.
• It's just wrong - use the prompt hint for vocabulary, and remember Whisper is not great with overlapping speakers, which is a model limitation, not a node bug.