(Deno) Audio Transcript

If you've ever wanted the lyrics or dialogue of an audio clip to steer your video prompt, you know the pain: transcribe somewhere else, copy the text, paste it into a prompt box, and hope it stays in sync when you tweak the clip. (Deno) Audio Transcript does the whole job inside the graph. Give it an AUDIO clip and it transcribes it locally with official OpenAI Whisper, then hands back the words plus a structured summary - language, timestamped segments, a confidence read - ready to feed a local LLM prompt director.

This node is the first half of Deno's beginner audio-reference workflow (the README's "MiniMax H3 R2V Audio Reference" lane): Whisper gets you the words, a Gemma 4 analysis gets you the sound, and a local LLM turns both into the actual video prompt. It's also just handy on its own whenever a graph needs text out of audio.

How it works

The name is honest about the big one: there's no API and no key. It runs the official openai-whisper package on your machine. Whatever AUDIO you feed in - mono or stereo - gets downmixed to mono and resampled to 16 kHz, which is what Whisper expects, then transcribed.

The first run downloads the Whisper checkpoint from OpenAI to ComfyUI/models/stt/whisper/ (checksum-validated by Whisper's own loader, so you're not trusting a random mirror), and it's cached there afterward. The node also does what it calls "CUDA Smart Swap": before Whisper runs it unloads ComfyUI's other models from VRAM, then releases Whisper when it's done - so a video model and Whisper don't fight over the same card. If openai-whisper isn't installed at all, you get a clear one-line install hint instead of a stack trace.

The inputs that matter

• audio - your source AUDIO. Grab it from any audio loader, like VHS Load Audio (Upload).
• model - large-v3-turbo by default (fast, and usually plenty). large-v3 buys accuracy on tricky speech and lyrics; medium and small trade accuracy for speed and VRAM.
• language - auto, or pin Korean / English / Japanese / Chinese if you know the clip and Whisper keeps guessing wrong.
• model_after_run - leave it on Unload after run unless you're transcribing in a loop; Keep loaded is the "I'm running many clips back to back" option.
• manual_transcript (optional) - your exact lyrics or dialogue. When non-empty it's authoritative for the wording, while Whisper still runs underneath to supply approximate segment timing.

Three outputs come out:

• audio_context - the structured STRING for an LLM: requested and detected language, a heuristic confidence band (built from Whisper's per-segment log probabilities), timestamped segments, all JSON-safe. This is what you wire into (Deno) Local LLM Loader's optional audio_context input.
• transcript - the plain text: your manual lyrics if you gave them, otherwise Whisper's output.
• audio - your original clip, passed through unchanged. That passthrough matters: it guarantees Whisper runs before the Gemma analysis step, so the Smart Swap unloads things in the right order.

Install

This is part of the Deno Custom Nodes pack, so install it once:

• ComfyUI Manager → search "deno custom nodes" → install → restart.
• Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/Deno2026/comfyui-deno-custom-nodes.git
cd comfyui-deno-custom-nodes
python -m pip install -r requirements.txt

openai-whisper is the pack's only declared Python dependency, and Manager installs it automatically. Restart ComfyUI afterward.

Where people get burned

The first run is the slow one - that checkpoint download plus model load can take a few minutes, and large-v3 is several GB. It's a one-time thing, cached under ComfyUI/models/stt/whisper/. If you see a "missing openai-whisper" message, you installed the clone without running requirements.txt (use the same Python that launches ComfyUI). Transcription needs torchaudio for the resampling step, which ships with ComfyUI's normal environment. And on CPU-only boxes it works, just slowly - the Smart Swap is CUDA-aware, but it can't invent a GPU you don't have. One more: if you're chaining this into the Gemma analysis step, feed the audio passthrough onward rather than re-loading the clip, so the swap order stays intact.