WanSoundImageToVideo

This is Wan 2.2 S2V - audio-driven cinematic video generation - in a conditioning node. Give it a reference image of a character and an audio track, and it produces a video where the character talks, sings, or otherwise moves in sync with the sound. Full body and half body, dialogue, singing, performance, whatever the audio says. It's the sibling to the plain WanImageToVideo that the community actually asked for: instead of prompting motion and hoping the mouth matches, the audio is the instruction. Alibaba shipped it in August 2025, and ComfyUI had native support the same week.

How it works

The audio goes through a separate wav2vec2 audio encoder into a per-frame audio embedding, which gets attached to the conditioning as audio_embed. The reference image gets VAE-encoded and attached as reference latents. During sampling the model cross-attends to the audio embedding, which is what drives lip sync and expressive motion. The text prompt still matters - it's the "what's happening in the scene" layer - but the audio is the motion source of truth.

The node also takes an optional control_video (a pose or motion reference clip, VAE-encoded and used as a control condition) and ref_motion (a reference video whose motion you want to imitate - capped at 73 frames, padded to that length if shorter). Skip both and it's plain image-plus-audio.

The inputs

• positive / negative - text conditioning. Describe the scene; the audio does the choreography.
• vae - the Wan 2.1 VAE (yes, 2.1 - the S2V model uses it, not the 2.2 VAE).
• width / height / length / batch_size - output geometry; 77 frames by default.
• audio_encoder_output - feed from AudioEncoderEncode, backed by a wav2vec2 audio encoder loaded from models/audio_encoders.
• ref_image - the character reference, required in practice.
• control_video, ref_motion - optional motion guides.

Outputs: positive, negative, and latent → KSampler. Conditioning node, not a generator.

Getting it

Ships with ComfyUI core. Model files: the S2V checkpoint (wan2.2_s2v_14B, fp8 or bf16) into models/diffusion_models, wav2vec2_large_english_fp16.safetensors into models/audio_encoders, plus wan_2.1_vae and the umt5 text encoder. Comfy-Org repackages them all.

Where people get burned

Length vs. audio. The official S2V code auto-sizes the clip to the audio; in ComfyUI you set length yourself. Push it past the audio's duration and the audio embedding runs to zeros - you get a clip that drifts into dead air. Match length to the audio. Wrong VAE. It wants wan_2.1_vae, and plenty of workflows grab the 2.2 VAE out of habit; sync dies or you get garbage. Negative conditioning is zeros by design - the node zeroes the audio embedding on the negative side rather than negating it, so don't expect CFG to fight the audio. And the video input is a pose/motion reference, not a content source - S2V isn't a vid2vid tool, it's image-plus-audio to video.