AuK Whisper Transcribe

Here's the thing people miss about speech editing: you can't write Replace 'Tuesday' with 'Friday' if you don't know exactly what words are in the recording, and you can't tell whether a lyric edit worked by listening once. This node solves both. It runs Whisper over any core AUDIO and hands you a plain transcript as a STRING.

Two honest uses, both from the README. Transcribe the source before writing a replace, insert, extract or lyric instruction, so the words you quote actually exist in the clip. Transcribe AuK's output as a quality check - whisper conversion in particular is easy to fool yourself about, since "quieter" isn't the same as "whispered." It can also be piped into AuK Prompt Enhance's context input, which gives the local language model the transcript of the source audio while it picks a task.

AuK itself never needs a transcript. Its encoder listens to the reference clip directly, unlike the reference-audio-plus-text pipelines elsewhere in the audio stack. This is an optional helper, not a dependency - you can delete it from the graph and nothing upstream breaks.

Inputs

• audio - required, core AUDIO type. Empty audio is rejected outright.
• model - a Whisper checkpoint name. The dropdown lists the six sizes it can fetch (tiny, base, small, medium, large-v3, large-v3-turbo) plus any folder containing a config.json under ComfyUI/models/whisper/ or ComfyUI/models/audio_encoders/.
• download_if_missing - default off, and that default is the point: with it off the node is fully offline and will error rather than quietly reach out. Flip it on and it fetches the checkpoint once from Hugging Face into models/whisper/<size>/, then reuses the local copy forever.
• model_size - the fallback size to fetch when the selected name isn't itself a known size. Default large-v3.
• language - spoken-language hint, with auto among the choices.
• task - transcribe keeps the spoken language, translate outputs English.

Output is a single STRING. It's an output node, so it previews the text in place and runs on its own when you queue - useful as a standalone QC step.

How it works

A Hugging Face Whisper model wrapped in ComfyUI's model management, running on the same device ComfyUI uses for text encoders, with offloading. Audio is downmixed and resampled to the processor's rate, and clips longer than 30 seconds go through Whisper's timestamp-guided long-form path rather than getting truncated - so an eight-minute interview transcribes in one pass. Progress is reported per frame, and the model is cached, which means one thing to know: the cache holds a single Whisper model, so two different sizes in the same workflow will reload each time.

Install

cd ComfyUI/custom_nodes && git clone https://github.com/Saganaki22/ComfyUI-AuK
cd ComfyUI-AuK && python -m pip install -r requirements.txt

Then either drop a Whisper folder with its config.json into ComfyUI/models/whisper/, or just enable download_if_missing and let it fetch one. large-v3-turbo is the sensible default for QC work - it's the size that gives you near-large accuracy at a fraction of the compute, which is exactly the profile you want for "did it say the words I hoped" checks. The download path uses huggingface_hub, which arrives with transformers, so there's nothing extra to install.

Troubleshooting

"Whisper model 'X' not found under .../whisper or .../audio_encoders." The folder has to be a directory containing config.json, not a bare .safetensors file sitting loose in models/whisper/. That trips up everyone who assumes this works like the diffusion loaders.

It downloaded anyway, or refuses to. The toggle is explicit and off by default precisely so an offline box stays offline. If the download failed halfway, the leftovers in models/whisper/<size>/ may be incomplete; delete the folder and retry rather than wondering why it half-works.

translate doesn't give you a French transcript. It gives you an English translation of French speech. If you want the original words, use transcribe and let language be auto or set it explicitly.

Numbers and names come out wrong. That's Whisper, not this wrapper. Transcribe with a bigger model for anything you're going to quote in an instruction, and treat a rough small-model transcript as a hint.

It costs VRAM at an awkward moment. The node defers to ComfyUI's text-encoder management, so it loads and unloads like an encoder rather than competing with your diffusion model at full tilt - but on a card that's already tight with a quantized AuK pair loaded, an extra large-v3 in the graph is a real cost. large-v3-turbo exists for that reason.

The author, Saganaki22, has shipped this kind of utility port repeatedly - the same pattern shows up in the Zonos2 and Higgs v3 TTS packs, where a working local path matters more than a polished one. This is that: modest, off by default, and it respects the fact that you might be running with the network unplugged.