XB_WanSoundImageToVideo

Wan 2.2 added audio-to-video: give it audio and it produces footage that syncs to it - a face whose lips move with the words, motion that hits the beats. This node is XB-BOX's conditioning/latent builder for that, with VAE tiling so it runs on cards that can't hold a whole video latent. It doesn't generate anything by itself; it takes your audio encoding, an optional reference image, and hands you conditioning plus a latent for the sampler to fill in. Display-wise it's the audio+image→video node in the pack's Pipeline category.

How it works

It calls the pack's shared xb_wan_sound_to_video helper. The audio path is the interesting part: your audio_encoder_output (from a Wan audio encoder, e.g. in the wrapper) carries per-layer encoded audio features at 50fps; the node interpolates those to the video frame rate, buckets them into 4-frame latent groups, slices out the range matching your length, and injects the result into the conditioning as audio_embed (the negative gets the same audio zeroed out, so the model learns what to not do). If you pass a ref_image, it's VAE-encoded into reference_latents - that's what keeps your character recognizable while it talks. Optional control_video and ref_motion (a short motion clip, capped at 73 frames) add structure. The output latent is a blank [batch, 16, (length-1)//4+1, h/8, w/8] - the sampler does all the actual work.

The inputs that matter

• positive / negative - your text conditioning.
• vae - the Wan video VAE.
• width (832) / height (480) / length (77) / batch_size - dimensions and frames. The defaults are the classic 16:9 Wan profile.
• vae_tile_size - tiling for the encode calls; drop it if you're OOMing.
• audio_encoder_output (optional) - the important one. Without audio, this is just an empty-latent image-to-video, so if you want the sound-synced behavior, this has to be wired.
• ref_image / control_video / ref_motion (optional) - identity, structure, and reference motion.

Outputs: positive, negative (with audio/identity injected) and latent - into a sampler, then a decode node.

Install

Pack install - ComfyUI Manager search XB_ToolBox, or clone into custom_nodes and restart. On top of that you need ComfyUI-WanVideoWrapper (this code calls its audio encoder types) and a Wan 2.2 model that supports audio in diffusion_models. The audio encoder node lives in the wrapper, and its AUDIO_ENCODER_OUTPUT plugs straight in here.

Common issues

The big one is expecting sound-synced output with no audio_encoder_output wired - you'll just get a generic image-to-video. Make your length match the audio's duration or the sync will drift by the end. And don't forget there's no audio in the output until you render with a video saver that can mux it; ComfyUI's save-video nodes handle that, but check your workflow's output stage.