Float Process

You've got a photo of a face and an audio clip, and you want the face to say the clip - mouth moving, head bobbing, some expression. That's the entire job of FloatProcess. It's the workhorse of the ComfyUI-FLOAT pack, the node that actually runs FLOAT (Generative Motion Latent Flow Matching for Audio-driven Talking Portrait, arXiv 2412.01064) and hands you back a sequence of image frames you can turn into a video.

The pitch, in one line: feed it one face image and one audio file, and it lip-syncs the face to the audio. It's one of the legit talking-portrait options in ComfyUI - community consensus puts it in the same conversation as Sonic and LivePortrait, and the usual verdict is "does its job, pretty good, nice and quick." The catch, and it's a real one, is in the crop flag below.

What it actually does

FLOAT is a flow-matching model: it learns a generative "motion latent" from the reference image plus the audio and flows it into talking frames. FloatProcess wires that up in ComfyUI terms. It takes your audio tensor, writes it to a temp WAV, saves your image to a temp PNG, then runs the model. The audio goes through a wav2vec2 encoder to get speech features, and the image gets face-detected, cropped to a square, and resized to 512×512 before it sees the model.

That last part is where people get surprised. Unless you set crop to true and the face is centered, the model itself only ever sees a face crop - the output video is basically the head region, not your full photo. You'll often see workflows that take the FLOAT frames and composite them back onto the original body. That's expected behavior, not a bug.

The inputs that matter

Most of the inputs are sensible once you know the vocabulary:

• ref_image - one image, and the node will hard-refuse a batch bigger than 1 ("Only a single image is supported"). Wire in a LoadImage directly.
• ref_audio - the speech you want synced. Long audio (3+ minutes, per the README) needs real RAM and VRAM headroom.
• float_pipe - the FLOAT_PIPE output from LoadFloatModels. Without it, nothing runs.
• a_cfg_scale - classifier-free guidance for the audio conditioning. Default 2, and that's a sane starting point; like any CFG dial, crank it and the model obeys the audio harder but gets more brittle. There's also r_cfg_scale (guidance on the reference image, default 1) and e_cfg_scale (intensity of the selected emotion, default 1 - the README suggests 5–10 if you want the emotion amped way up).
• emotion - none, angry, disgust, fear, happy, neutral, sad, surprise. This is real: the pack ships a wav2vec2 emotion-recognition encoder and uses it to bias the motion. "none" skips it entirely.
• crop - enable this only if your reference image doesn't have a centered face. It runs face detection first and crops around the detected face.
• fps and seed - output frame rate (default 25) and reproducibility.

Output and wiring

The single output is images (an IMAGE tensor of frames). The pack's own example workflow pipes that into Video Helper Suite's Video Combine to mux the audio back in - which you'll want, because FloatProcess gives you frames, not a sound-on MP4. Note that the node returns frames normalized 0–1, ready for the usual video nodes.

Install and gotchas

Install the pack once - both nodes ship together:

cd ComfyUI/custom_nodes
git clone https://github.com/yuvraj108c/ComfyUI-FLOAT
cd ComfyUI-FLOAT
pip install -r requirements.txt

ComfyUI Manager finds it under "ComfyUI FLOAT" if you prefer. The requirements list is heavy - face_alignment, torchcodec, librosa, timm, transformers<5.0.0 - so expect a beefy first install and watch for version clashes with other custom nodes. The first queue also downloads the models (~the whole HF repo yuvraj108c/float) into ComfyUI/models/float, so the first run is slow even before inference.

One honest heads-up the README itself gives you: the author points to ComfyUI-FLOAT_Optimized as a more advanced, maintained version. If FLOAT becomes a regular part of your workflow, that fork is worth a look - it fixes some of this pack's rough edges. And if your character isn't a clean, front-facing headshot, expect to composite the talking head back onto the body yourself; FLOAT's face-crop output doesn't do full-body talking.