ITL Whisper Loader

Speech-to-text has quietly become the most useful boring thing in a ComfyUI audio graph. The headlining reason in this pack is that Breeze TTS voice cloning needs a verbatim transcript of your reference clip, and hand-typing one is how you end up with a mediocre clone. The broader reason: transcription is a real job in its own right - captions, transcripts for a script, turning a voice memo into text you then feed back into the graph as a prompt.

This node is just the loader. Two dropdowns, one output: a model socket of the pack's own ITL_WHISPER type, which wires into ITL Whisper Transcribe.

The two dropdowns

model gives you six official OpenAI Whisper checkpoints, in this order: large-v3-turbo (the default), large-v3, medium, small, base, tiny. The tooltip is honest about the tradeoff - turbo is the best speed/quality balance, large-v3 is the most accurate, smaller is faster and worse. Two things the sizes don't tell you: medium weighs in at 3.1 GB, the same download as large-v3, while being a smaller and less capable model, so there's no reason to pick it for disk reasons. And large-v3-turbo is 1.6 GB at 809M parameters against large-v3's 3.1 GB at 1.55B.

device is auto, cuda, or cpu. auto means CUDA when it's there, CPU otherwise. Put it on cpu deliberately if you want your GPU entirely free for a render while transcription grinds away in the background - this is the one part of the pack's audio suite that runs without an NVIDIA card, unlike Breeze. It runs fp16 on CUDA and fp32 on CPU.

What it does behind the scenes

On first use it downloads the checkpoint from Hugging Face into models/whisper/whisper-<model>/ - and politely only the eleven files it needs: the safetensors weights, configs and tokenizer. The large-v3 repos also carry Flax, TensorFlow, .bin and fp32 copies that add up to roughly 12 GB, none of which get pulled. Downloads resume if interrupted, and the loader checks for missing or 0-byte files before loading, so a killed download re-fetches instead of failing mysteriously later.

It then loads the model and emits a small cache key as its output - not the model object. That's a deliberate design choice repeated across this pack: the graph carries a key so ComfyUI's output cache never holds weights, and the pack's internal cache keeps the model resident so your second transcription doesn't pay a reload. One Whisper model stays resident at a time; two loaders with different models in one workflow reload on every run, which is worth avoiding.

The output type is deliberately namespaced ITL_WHISPER rather than a generic WHISPER, because ComfyUI matches links by type name globally and some other pack's WHISPER payload would plug in happily and then fail to unpack. You'll only ever wire this into this pack's transcribe node, and it won't let you make that mistake.

Install

ComfyUI Manager → search ComfyUI-IntoTheLatent-Utils → install → restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/Into-The-Latent/ComfyUI-IntoTheLatent-Utils
pip install -r ComfyUI-IntoTheLatent-Utils/requirements.txt

Whisper itself needs only transformers (shipped with ComfyUI, and this pack pins >=4.57,<6) - the pack's requirements add librosa for resampling, plus soundfile, numpy and pillow. If the nodes say a dependency is missing, run Manager's Try fix on this pack.

Gotchas

VRAM is modest. large-v3-turbo sits around 2 GiB in fp16, large-v3 around 3.5 GiB. Whisper and Breeze can be resident at the same time for roughly 10 GiB total, so a workflow that transcribes a reference and then clones it doesn't thrash.

Pick the right size for the audio, not for your ego. For clean, single-speaker speech - which is what a reference clip should be - large-v3-turbo is plenty and much faster. large-v3 is for the case where turbo is genuinely mishearing words in noisy or accented audio, or where you need its better long-form handling on a messy recording.

cuda with no CUDA errors out rather than silently falling back. That's the right behaviour; if you're on a CPU-only box, set auto or cpu.

Real-world note: whisper model names barely register in the image-generation community's discussion, so there's not much folklore to lean on here - large-v3-turbo shows up about once in recent community threads. Judge it on your own transcripts, not on consensus.