Higgs v3 Whisper Transcribe

This node doesn't make any speech, and that's the point. Higgs v3 Whisper Transcribe is the pack's support act: it takes a reference AUDIO clip and hands back a STRING transcript that you feed into Voice Clone's reference_text - or Multi-Speaker's per-speaker transcript slots. It exists because a correct reference transcript is the single biggest quality lever in zero-shot cloning, and typing one out by hand every time is miserable. Run the clip through Whisper, wire the output over, done.

It's a plain Whisper pipeline under the hood - the whisper-large-v3-turbo model by default, run through Transformers, with models stored in ComfyUI/models/audio_encoders/. Nothing exotic. The first run downloads the Whisper weights (turbo is a few GB; whisper-tiny is much smaller if you just need a rough draft), and from there it's the usual transcribe-or-translate ASR business.

The inputs that matter

• audio (required) - the same reference clip you're feeding Voice Clone. Clean speech transcribes better, which conveniently is also what clones better.
• model - five choices from whisper-tiny up to whisper-large-v3, defaulting to large-v3-turbo. Turbo is the sane default: fast, accurate enough for reference transcripts. Drop to a smaller model if you're only generating a first draft.
• language - auto detects it; setting it explicitly (english, chinese, japanese, ...) improves accuracy on accented or noisy clips.
• task - transcribe keeps the source language; translate returns English. The latter is handy when you want to clone from a clip in a language you don't read.
• chunk_length_s - Whisper's chunk window, default 30s. Leave it unless you have long clips; 0 lets Transformers decide.
• dtype / download_if_missing - precision (auto is bf16 where supported) and auto-downloading the selected Whisper model.

Output: a single transcript (STRING). Wire it straight into reference_text on Higgs v3 Voice Clone.

Where it helps and where it doesn't

Here's the honest part, straight from the pack's own README: "Whisper can help, but a manually corrected transcript is better." Whisper gets most clips right and will absolutely get you a usable clone, but ASR errors - wrong words, dropped punctuation, invented homophones - quietly degrade the speaker conditioning, and there's no error message telling you it happened. So treat the output as a draft. For short reference clips, proofreading 10 seconds of text takes thirty seconds and meaningfully tightens the clone. For multi-speaker setups, run and correct each speaker's clip separately so every transcript matches its own voice.

One more footgun while you're here: Whisper is registered with ComfyUI model management, but it's a separate chunk of memory on top of the ~11 GB Higgs model. On a mid-range card, don't transcribe and generate in the same eager queue and wonder why VRAM spikes. Do the transcription first, stash the transcript, then run generation - or just accept the swap.

It's a small node and a thin one, but it's the difference between "cloning is fiddly" and "cloning is a two-node workflow that mostly just works." That's worth the install on its own.