Load FLOAT FMT Model (VA)

FLOAT = "Flow Matching Transformer," and that transformer is the actual brain of the whole talking-head system. The synthesis model (see Load FLOAT Synthesis (VA)) knows how to render a face, but it's useless until something tells it how the face should move, frame by frame. That something is this node: it loads the FMT weights - the module that maps audio, identity, and emotion into a sequence of motion vectors.

Like the other Very Advanced loaders, this one exists for people pulling FLOAT apart. If you're on the regular workflow (Load FLOAT Models (Opt) + FLOAT Process (Opt)), you never touch it. If you're in the VA workflow, it's the last and most interesting loader in the chain, and its output feeds the ODE sampler.

The mechanism

Here's the core idea of flow matching, compressed: instead of learning to denoise like a diffusion model, FLOAT learns a vector field - at any moment t from 0 to 1, given the current state and the conditioning (audio, identity, emotion), it predicts which direction the motion should flow. Then a solver integrates that field from noise at t=0 to a finished motion sequence at t=1. That integrator is torchdiffeq (the odeint call), and this is where the pack's dependency on torchdiffeq in requirements.txt comes from.

The loader reads fmt.safetensors from models/float/fmt/ and infers the parts of the architecture you shouldn't have to specify: hidden dim (dim_h), the transformer depth (fmt_depth), and the MLP ratio - all read straight off the weight shapes. Then it asks you for the parameters it can't infer, because they define the temporal structure, not the weights:

• fps - frames per second of the video you're generating (default 25). This shapes the time grid.
• wav2vec_sec - seconds of audio processed per chunk (default 2.0), defining the audio→frame correspondence.
• num_prev_frames - how many previous frames are used as context (default 10).
• attention_window - the local attention mask window size (default 2): frame t attends to t−1, t, t+1.
• dim_e - dimension of the emotion latent, which should equal the number of emotion classes from your emotion model (default 7).
• num_heads - attention head count (default 8), an architectural constant that must match the checkpoint.

The defaults come from the pack's base options, and they're correct for the stock model. The node validates your values against the loaded checkpoint, so a wrong num_heads or dim_e usually errors loudly instead of silently producing garbage. That's a feature.

Outputs

• float_fmt_model (FLOAT_FMT_MODEL) - the loaded transformer, wired into FloatSampleMotionSequenceRD_VA, the node that runs the ODE solver and produces the driven motion sequence.
• fps (FLOAT) - echoes your setting so downstream nodes can stay in sync.
• fmt_options_out (ADV_FLOAT_DICT) - the effective options, which you can pipe into other FLOAT nodes.
• conditioning_chunk_size (INT) - the computed chunk size for conditioning, used by the sampler to align audio and frames.

Getting the weights

Same story as the other VA loaders: if fmt.safetensors isn't in models/float/fmt/, the node tries to extract it from the unified FLOAT.safetensors, and only downloads the component directly if that's missing too. So the 2.4 GB unified file still gets you everything, including the FMT.

Notes

• The defaults are fine. If you're just following the VA example workflow, change nothing here. fps is the one you might legitimately touch (25 → 30 to match your screen, at a memory cost).
• Don't conflate dim_e with emotion intensity. It's the class count of the emotion model you loaded - it has to match that model's output, not your mood.
• FMT is a transformer with local attention, and the VA workflow is explicitly the lower-VRAM path through FLOAT. If you're hitting memory walls on the regular nodes, this is where to investigate.

The line that ties it together: Load FLOAT Synthesis (VA) renders the face; Load FLOAT FMT Model (VA) decides what the face is saying. Audio in, motion out - that's the whole game.