Load Wav2Vec Model (for Audio Encoding) (VA)

Every talking-head model needs ears, and FLOAT's ears are Wav2Vec 2.0 - Facebook/Meta's audio encoder, the same one that powers a chunk of speech-recognition research. This Very Advanced (VA) node loads one of those models from a HuggingFace-style folder and wraps it for FLOAT's specific needs. The audio that comes out is the content signal - what's being said, the pacing, the cadence - which eventually becomes the wa_latent that drives the mouth.

If you're using the regular FLOAT workflow, stop here - Load FLOAT Models (Opt) already bundles the wav2vec weights inside the unified 2.4 GB file and you never see this node. This loader exists only for the VA workflow, where each network is loaded separately so you can swap parts. It's the first node on the VA graph's audio path.

What it actually loads

The model_folder dropdown lists directories inside ComfyUI/models/audio/. The default is wav2vec2-base-960h (Facebook's, Apache 2.0 licensed), and you can point it at any Wav2Vec2-type model with the right layout. The loader needs the folder to contain the JSON config files plus weights - model.safetensors, or the older pytorch_model.bin as a fallback. The README explicitly says you don't need pytorch_model.bin or tf_model.h5 if you have the safetensors; you just need the config JSONs and the weights.

Here's the part that's easy to miss: it doesn't just hand you a stock Wav2Vec model. It wraps it in FLOAT's own FloatWav2VecModel subclass, which adds time-domain interpolation. Stock wav2vec features land on a fixed frame grid that never lines up with your video's frames; the wrapper resamples the audio features so they map cleanly onto the video timeline. That's the detail that makes lip-sync actually work instead of drifting.

Under the hood it loads the config with attn_implementation="eager" - a deliberate choice. The pack's changelog notes a fix for broken Transformers installs where the default attention implementation was incompatible, and this is it. If your wav2vec loading ever misbehaves after a transformers update, this is the line to know about.

Inputs

Just two:

• model_folder - which audio model folder to load from, defaulting to wav2vec2-base-960h.
• target_device - where the weights go (CPU default, CUDA if you're chasing speed).

That's the whole list. The node is deliberately boring, and that's good.

Outputs

• sampling_rate (INT) - the rate the loaded model expects (16000 for the base model), so downstream nodes know how to feed it audio.
• wav2vec_pipe (WAV2VEC_PIPE) - a tuple of the wrapped model plus its feature extractor. This wires into FloatAudioPreprocessAndFeatureExtract, the node that chops the audio into chunks, runs it through this model, and produces the features the rest of the VA graph consumes.

Getting the weights

If the folder is missing entirely, you'll get a clear error telling you to put HF model folders into models/audio/. Two escape hatches exist: if the unified FLOAT.safetensors is present, the node extracts the wav2vec2_base part from it; otherwise it downloads the component directly. So the VA graph can still bootstrap from the one 2.4 GB download - it just splits things up afterward.

Don't confuse it with the emotion model

Wav2Vec is the content ears. FLOAT also has an emotion recognizer - also wav2vec-based (a fine-tuned wav2vec2-large-xlsr-53-english), also lives under models/audio/, but it's loaded by a different node (LoadEmotionRecognitionModel) and its output is the dim_e emotion classes that Load FLOAT FMT Model (VA) needs. Two wav2vec models, two loaders, two jobs: one hears the words, the other hears the feeling. Mix them up and your talking head will lip-sync fine but emote wrong.

The one-sentence version: this node gives FLOAT an ear, tuned to land its features on the video's frame grid, so the words in the audio can become mouth shapes in the picture.