Whisper Model Loader

Whisper is OpenAI's speech-recognition model, and this node loads it inside the WanVideoWrapper. Where wav2vec2 gives you raw audio features for driving mouth motion, Whisper gives you transcription and timing - it knows what words were said and roughly when. In Wan's audio-driven video workflows (talking heads, lip-sync, InfiniteTalk-style speech), that word-and-timing information is useful for aligning the generated motion to speech. This node's job is simply to get Whisper into memory and hand it downstream as a WHISPERMODEL.

If you found this page because a talking-head workflow has a Whisper loader in it and you weren't sure why speech-to-text belongs in a video graph - that's it. It's there for timing and alignment, not for captions.

How it works

Whisper takes an audio clip and produces a transcription along with timestamps. In an audio-driven Wan pipeline, that timing scaffold helps the downstream stage line up mouth and head motion with the actual speech. This node loads the Whisper weights at the precision and device you choose and exposes the loaded model; the transcription/alignment work happens in the nodes that consume it.

The inputs and outputs that matter

• model (enum) - which Whisper model to load, listed from your local models folder. An empty dropdown means no Whisper model is downloaded yet - the usual first-run stumble.
• base_precision (fp16 default, or bf16 / fp32) - load precision; fp16 is the light default.
• load_device (main_device default, or offload_device) - keep on main_device for speed, offload if VRAM is tight.

Output is whisper_model (WHISPERMODEL) - wire it into the audio-driven stage that asks for it.

How to install it

Ships in the WanVideoWrapper. ComfyUI Manager: search ComfyUI-WanVideoWrapper, install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-WanVideoWrapper
pip install -r ComfyUI-WanVideoWrapper/requirements.txt

then restart. You need a Whisper model downloaded into your models folder for the dropdown to populate. Whisper comes in sizes (tiny through large) that trade accuracy for speed and VRAM - a smaller one is usually plenty for driving video timing, so don't reach for large reflexively.

Common issues & troubleshooting

Empty dropdown. No Whisper weights found. Download a model into the right folder and refresh. Standard first-run issue for every loader in this pack.

Whisper vs wav2vec2 - which do I need? They're not interchangeable. wav2vec2 produces the per-frame audio features that actually drive mouth motion; Whisper produces transcription/timing. Depending on your workflow you may need one, the other, or both - follow the example graph for your specific audio-driven model rather than assuming.

It's slow to load a large model. Whisper large is heavy and rarely necessary here. Drop to a smaller size; for lip-timing you almost never need transcription accuracy that fine, and you'll claw back both load time and VRAM.