Multi/InfiniteTalk Wav2vec2 Embeds

Talking-head video is one of the things Wan does genuinely well through the MultiTalk and InfiniteTalk pipelines, and this node is where the audio turns into something the model can lip-sync to. It runs your audio through a Wav2Vec2 speech model and produces the embeddings that drive mouth movement. The "Multi" is the good part: it takes up to four separate audio tracks plus reference masks, so you can have two (or more) characters in one shot each speaking their own line, instead of one talking head at a time.

For continuous talking-head content specifically, this is the path people reach for - InfiniteTalk (from the MultiTalk team) is widely rated above Wan's own S2V for this job, and it runs on top of the same audio-embeds mechanism.

How it works

Wav2Vec2 is a speech representation model - it encodes raw audio into features that capture phonetic content over time. This node runs your AUDIO through the loaded wav2vec_model, aligns it to your target frame count and fps, and emits MULTITALK_EMBEDS that the sampler uses (via the extra-args bundle) to shape lip and face motion. With multiple audio inputs it assigns each to a masked region so different people get different mouths.

The inputs and outputs that matter

• wav2vec_model (WAV2VECMODEL) - the loaded Wav2Vec2 model; you load this with a separate loader node upstream.
• audio_1 - the primary voice track. audio_2 … audio_4 (optional) - additional speakers.
• num_frames (default 81) and fps (default 25) - set these to match your clip. The audio gets aligned to this timing, so a mismatch is where lip-sync drifts.
• audio_scale and audio_cfg_scale (both default 1) - how strongly the audio drives motion, and its guidance weight. Turn up if the mouth's barely moving.
• ref_target_masks (MASK, optional) - which region each speaker's audio controls. This is what makes multi-person talking work.
• multi_audio_type (para / add) - whether multiple tracks play in parallel (para) or are summed.
• normalize_loudness (default true) - evens out audio level before encoding; leave it on unless you've already normalized upstream.

Outputs: multitalk_embeds (into WanVideoSamplerExtraArgs), the possibly-processed audio (mux it back onto the final video), and num_frames.

How to install it

ComfyUI Manager - search ComfyUI-WanVideoWrapper, install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-WanVideoWrapper
pip install -r ComfyUI-WanVideoWrapper/requirements.txt

then restart. You'll need the Wav2Vec2 model and the MultiTalk/InfiniteTalk weights - both live on Kijai's WanVideo_comfy HuggingFace repo - plus a compatible Wan base.

Common issues & troubleshooting

Black screen output. This is a known InfiniteTalk-workflow snag people hit and post about; it usually traces to a mismatched model file or a wiring gap in the audio chain rather than this node itself. Double-check the Wav2Vec2 and talk-model files are the ones the workflow expects.

Lip-sync drifts out over the clip. Almost always a timing mismatch - your fps and num_frames here have to agree with what the sampler actually renders. Set them from the same source and they'll stay locked.

Two speakers, one mouth moving. You need ref_target_masks to tell the model which region each audio track drives. Without masks, multi-audio has nowhere to route the extra voices.

It's slow. Talking-head runs are long - an InfiniteTalk clip can push well past an hour on a 4090 for a decent length. That's the pipeline, not this encode step.