FLOAT Process (Opt)

This is the node you came for. FLOAT Process (Opt) is the whole FLOAT talking-head pipeline stuffed into a single box: give it a photo of a person, an audio clip, and the float_pipe from Load FLOAT Models (Opt), and it returns a video of that person's lips, jaw, cheeks, and head moving in sync with the audio. That's it. No encoder chains, no separate samplers, no mid-pipeline latents to worry about.

FLOAT - Generative Motion Latent Flow Matching for Audio-driven Talking Portrait, a DeepBrain AI Research paper (arXiv 2412.01064) - is the model underneath. It's in the same family as InfiniteTalk, Wan S2V, and the rest of the audio-driven avatar crowd, but it's unusual in how cleanly it separates appearance from motion. The reference image is encoded into "this is what they look like" (appearance latents) and "this is the pose they're in right now" (a motion vector). The audio is turned into a conditioning sequence, and a Flow Matching Transformer (FMT) then samples a whole sequence of motion vectors that plausibly follow the voice. The decoder finally renders each frame from the appearance + that motion. It's diffusion-adjacent but learns a flow field instead of a denoising process, which is why you see ODE solver settings floating around this pack rather than steps and CFG the way you know them from SD.

The inputs that matter

• ref_image - a square image, ideally 512×512 (that's what the net trained on; anything else gets rescaled). Simple backgrounds, face not too small, and leave headroom for motion - or just leave face_align on (it's true by default) and it'll crop around the detected face for you.
• ref_audio - the voice to drive it. Mono and clean beats stereo and musical. The emotion detector was trained on English, so keep that in mind; the output video is exactly as long as the audio.
• float_pipe - from the Load FLOAT Models (Opt) node. This is where the models live.
• a_cfg_scale (default 2) and e_cfg_scale (default 1) - audio and emotion guidance. Crank audio up if the mouth lags or over-pronounces; the tradeoff is more over-animated motion.
• emotion - pick one of angry, disgust, fear, happy, neutral, sad, surprise, or leave none to let the model guess from the voice.
• fps (default 25), seed (set it fixed for repeatable results), face_align.

The outputs are refreshingly boring: images (the frame sequence), a passthrough ref_audio, and fps so a downstream video-assembly node knows the timing.

Install

ComfyUI Manager (search "ComfyUI-FLOAT_Optimized"), or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/set-soft/ComfyUI-FLOAT_Optimized
cd ComfyUI-FLOAT_Optimized
pip install -r requirements.txt

Then restart ComfyUI. First run is the gotcha: the models auto-download to ComfyUI/models/float - the unified FLOAT.safetensors is 2.4 GiB, and the Load FLOAT Models (Opt) node will sit there looking dead while it pulls. Watch the console for progress. The author tests on an RTX 3060 (12 GB VRAM, 32 GB RAM + swap), so it's built for modest hardware, but long audio means more frames means more memory.

Common issues

The model is CC BY-NC-SA 4.0 - non-commercial, and the README flags it in bold. Don't build a paid product on it. On the practical side, most "why is the mouth wrong" complaints trace back to the reference image (non-square, busy background, face at the edge) or the audio (music left in, stereo, clipped speech). And if the first inference is brutally slow, that's the model loading, not a hang.

If you outgrow this one-button version, the same pack ships FLOAT Advanced Options and the VA nodes so you can drive the ODE solver and every condition by hand. But for "make this person say this," this node is the whole answer.