LongCat Avatar Whisper Embeds (v1.5)

This is the front door to a talking-avatar workflow in Kijai's Wan wrapper. You feed it an audio clip (or several) and a Whisper model, and it hands back an embedding that tells the LongCat Avatar model how the face should move to match the sound. Think lip-sync plus head motion plus a bit of emotion, driven entirely off the waveform.

The name is a little dense, so unpack it: LongCat is the model family (LongCat is a very active line right now - the ComfyUI crowd has been all over it), Avatar is the audio-to-talking-head task, and Whisper Embeds means it uses OpenAI's Whisper speech model to turn your audio into features the video model can read. This node doesn't generate video. It builds the conditioning; the actual frames come out of the Wan sampler downstream.

How it works

Raw audio is useless to a diffusion model - it needs the content of the speech, not the bytes. So the node runs your clip through a Whisper model to get per-frame audio features, then packages them at your target frame rate and length. The output is typed MULTITALK_EMBEDS, which is the tell: this rides the same rails as the MultiTalk talking-head system, so it slots into that sampler path and can drive more than one speaker at once.

That multi-speaker bit is the interesting part. You get four audio slots, and multi_audio_type decides how they combine - para (parallel) treats them as separate people talking at the same time, while add layers them. Pair that with ref_target_masks to say which face each audio track belongs to, and you can animate two characters in one shot having a conversation.

The inputs and outputs that matter

• whisper_model and audio_1 - the two required feeds. The Whisper model comes from a loader node in the pack; audio_1 is your speech clip.
• num_frames (default 93) and fps (default 25) - set your clip length and rate here. These must line up with what you ask the sampler to render, or your lip-sync drifts. Note the output also returns num_frames, so you can wire it straight into the sampler and keep them locked together.
• audio_scale / audio_cfg_scale - how hard the audio pushes the motion. Turn these up if the mouth is barely moving; ease off if the face is over-acting.
• normalize_loudness (on by default) - evens out quiet vs loud audio so the drive strength is consistent. Leave it on unless you've already normalized.

Outputs are multitalk_embeds (into the LongCat/MultiTalk sampler), the processed audio (so you can mux it back onto the final video), and num_frames.

How to install it

Either route works. In ComfyUI Manager, search ComfyUI-WanVideoWrapper, install, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-WanVideoWrapper
pip install -r ComfyUI-WanVideoWrapper/requirements.txt

then restart. Beyond the node pack you need three model families on disk: the LongCat Avatar video weights (Kijai republishes Wan-adjacent models as fp8 under huggingface.co/Kijai/WanVideo_comfy - grab the fp8 build unless you have VRAM to burn), a Whisper model for the audio encoding, and the usual Wan VAE and text encoder. Drop them in the matching ComfyUI/models subfolders.

Common issues & troubleshooting

The node is red / "missing nodes." LongCat Avatar is recent, so an older copy of the wrapper simply won't have this class. Update the pack (Manager → update, or git pull in the folder) and restart - that's the single most common report on new WanVideoWrapper nodes.

Lip-sync drifts out over the clip. Almost always a frame-count or fps mismatch between this node and the sampler. Keep num_frames and fps identical across both, and remember Wan's native ceiling is 81 frames per pass - long audio means chunking, and sync error accumulates across chunks.

Mouth barely moves, or moves like a puppet. That's the audio_scale / audio_cfg_scale pair. Nudge them together rather than one at a time.

Two speakers, one talking mouth. You didn't route ref_target_masks, so the model can't tell which face owns which audio. Mask each subject and feed both.