FLOAT Encode Audio to latent wa (Ad)

A talking head is a face that moves to sound, so somewhere in the pipeline the audio has to become a signal the motion model can condition on. FLOAT Encode Audio to latent wa (Ad) is that step: it takes the raw audio, resamples and preprocesses it using the float_pipe's internal processor, runs it through the Wav2Vec 2.0 encoder, and projects it down into wa_latent - the audio conditioning sequence that drives lip and head motion.

In the flow, this is the "voice" branch. The wa_latent it outputs is the primary driver for the Flow Matching Transformer: when the sampler generates the motion sequence, it's essentially asking "given this person, this audio, and this emotion, what motion makes the lips match this sound?" No wa_latent, no sync.

Inputs and outputs

Three inputs, and only one of them needs your attention most of the time:

• float_pipe - from Load FLOAT Models (Opt); it carries the Wav2Vec model and projection layer internally.
• audio - the ComfyUI AUDIO input. This node resamples and normalizes it for you, which is nicer than the Very Advanced path where you're expected to hand over pre-validated mono audio at the right sample rate.
• fps (default 25) - the target video frame rate. It matters more than people expect: the audio features are interpolated to match this FPS, so the output video length and the number of generated frames both derive from it. Change fps between runs and you change the frame count.

Outputs: wa_latent (the audio conditioning latent, feeds FLOAT Sample Motion Sequence rd), audio_num_frames (total video frames from audio length × fps - the sampler needs this exact number so its output matches the audio), processed_audio_features (features before the Wav2Vec model - these are what the emotion encoder wants), and a float_pipe passthrough.

That processed_audio_features output is the quiet key to the Advanced workflow: it's what you feed into FLOAT Encode Emotion to latent we so the emotion prediction is derived from the same audio features, keeping the two conditioning signals consistent.

Practical notes

The quality of your wa_latent is the quality of your audio. Clean, mono speech wins over music-laden stereo every time - the README is blunt that if the input is a song, remove the music. Long audio also means proportionally more frames and more VRAM, since the generated video is exactly as long as the clip.

Install is the usual pack routine - ComfyUI Manager search "ComfyUI-FLOAT_Optimized", or git clone https://github.com/set-soft/ComfyUI-FLOAT_Optimized into custom_nodes, pip install -r requirements.txt, restart, first-run model download (~2.4 GiB to models/float). One license note worth repeating since it's the whole pack's: FLOAT is CC BY-NC-SA 4.0 - fine for personal and research use, not for commercial products. And if your output video comes back the wrong length or the sampler rejects the frame count, check that fps here matches the fps everywhere else in the graph; mismatched frame counts are the classic Advanced-workflow failure.