LongCat Avatar Audio Encode

LongCat_Video_SM_Audio is where your audio clip becomes the conditioning that drives the avatar's mouth. Whisper-large-v3 gives you a model; this node gives you the embeddings - the au_cond that the sampler latches lip movement onto. It's the audio twin of the text encode node, and it's the part of the pipeline that makes LongCat genuinely different from image-to-video models that just animate in the general direction of a track.

How it works

The node runs your audio through the Whisper-large-v3 encoder that LongCat Avatar Whisper loads, producing full-clip audio conditioning. It computes the segment count automatically from the audio length and the current save_fps, so you don't set "how many windows" anywhere - feed it a clip and it figures out the frame math. At 25 fps an 18-second clip computes to 6 segments, because the first window is 93 frames and each continuation adds 80 (13 overlap frames are kept for continuity). The sampler trims the generated envelope back to your source audio's frame count at the end, so you don't get extra silent frames.

The inputs

• audio_encoder - the AUDIO_ENCODER output from LongCat Avatar Whisper. Nothing else works; this node is Whisper-bound for Avatar 1.5.
• audio - the AUDIO you want the avatar to speak. Comes from LoadAudio, or from the vocal-extract/crop chain if you're working from a full song.
• save_fps (default 25) - do not touch this lightly. It feeds window segmentation and output frame counts, and the README is blunt: non-25 fps values made lip sync and motion timing unstable in local validation.
• audio_type - para (simultaneous two-person dialogue, both clips should be the same length) or add (sequential turn-taking - left audio plays first, then the right).
• left_audio (optional) - connecting this switches to two-person mode. The naming is about screen position, not stereo channels: left_audio = the person on the left side of the frame, audio = the person on the right side.
• p_box (optional) - comma-separated person boxes telling the node where people are in the original image, in [y_min, x_min, y_max, x_max] order, left person first. Empty still builds conditioning, just without explicit location hints. The boxes get resized and center-cropped together with the image for inference, so measure on the source image, not the generated output.

Output is a single au_cond (CONDITIONING) that goes into the sampler's au_cond socket.

Install

This is one node in the ComfyUI-LongCat-Avatar pack - install once for all nine:

cd ComfyUI/custom_nodes
git clone https://github.com/rookiestar28/ComfyUI-LongCat-Avatar
cd ComfyUI-LongCat-Avatar
pip install -r requirements.txt

or ComfyUI Manager → search ComfyUI-LongCat-Avatar → restart. NVIDIA + CUDA required.

Where people get burned

The big one: avatar version mismatch. This node is Whisper-only for Avatar 1.5 - if you connect a Wav2Vec2 encoder (the Avatar 1.0-era path), the Whisper node itself will reject it, so this node won't even see it. Good guardrail, don't fight it. Second: in two-person mode people connect left_audio and audio by stereo channels and get confused when the wrong person's lips move. Think screen left / screen right. Third: if your avatar's lips lag or drift, the first suspect is save_fps - keep it at 25. If the source clip has a song over the voice, run it through LongCat Avatar Vocal Extract first so the model is syncing to the voice, not the instruments.