FLOAT Apply Audio Projection (VA)

The Very Advanced audio path is a two-stage squeeze: FLOAT Audio Feature Extract runs the voice through Wav2Vec and gets big, high-dimensional features back; FLOAT Apply Audio Projection (VA) is the second stage that projects those down into wa_latent, the compact audio conditioning tensor that lives in FLOAT's motion-latent space. Two small inputs, one tensor out, and it's the last thing standing between "an audio file" and "a signal the Flow Matching Transformer can steer lips with."

Why the projection exists at all is the interesting part. Wav2Vec features are rich - a few hundred dimensions per time step - but the FMT was trained to condition on a specific, much smaller audio embedding (the dim_a your projection layer was built for). The projection layer is the trained bridge between those two spaces. It's also the seam where the pack's modularity shows: because this node takes an explicit projection_layer, you could in principle swap in a projection trained for a different audio encoder and watch the whole conditioning path change. That's a Very-Advanced-grade experiment, but it's exactly what this graph is for.

Inputs and outputs

• wav2vec_features - the interpolated feature tensors from FLOAT Audio Feature Extract.
• projection_layer - from Load Audio Projection Layer, which reads projection.safetensors from models/float/audio_projections/.

Output: wa_latent - the audio conditioning latent, ready for Float Sample Motion Sequence RD VA. That's the whole job: one wire in the VA graph that connects the audio branch to the sampler.

There's a subtle dependency to respect: the projection layer was built with a specific input dimension, which the loader infers from the weights (inferred_input_dim). If you later flip only_last_features in the audio-feature node - which changes the feature dimension drastically - the projection will no longer match. Use the loader's inferred dimensions as the contract for the audio branch, and keep them in sync.

Install & the usual caveats

Pack install, as ever: ComfyUI Manager search "ComfyUI-FLOAT_Optimized", or cd ComfyUI/custom_nodes && git clone https://github.com/set-soft/ComfyUI-FLOAT_Optimized && pip install -r requirements.txt, restart. The projection weights auto-extract from the unified FLOAT.safetensors (prefix audio_encoder.audio_projection) or download as projection.safetensors if that's missing. If the output video comes back with lips that never quite sync, the audio features and the projection are the first place to look - wrong dimension, wrong rate, or features from a mismatched Wav2Vec layer. And the pack-wide reminder, since it's easy to miss in the excitement of a working talking head: FLOAT is CC BY-NC-SA 4.0, non-commercial.