Nodes/ComfyUI-FLOAT_Optimized/Load FLOAT Synthesis (VA)
ComfyUI Node

Load FLOAT Synthesis (VA)

The renderer half of FLOAT's motion autoencoder

By set-soft·Created about a year ago·Updated 12 months ago· 33
Load FLOAT Synthesis (VA)
    • float_synthesis
    • inferred_size
    • inferred_style_dim
    • inferred_motion_dim
    synthesis_file
    target_devicecpu
    channel_multiplier1
    blur_kernel_str[1, 3, 3, 1]
    cudnn_benchmarkfalse

    FLOAT's trick is that it separates what a person looks like from how they move. The "what they look like" half lives in a motion autoencoder: an Encoder that reads your reference photo, and a Synthesis module (basically the decoder) that renders frames. This node is the loader for that decoder, and it's part of the pack's "Very Advanced" (VA) workflow.

    Before you go deep here, a reality check. If your goal is "make this photo talk," you don't want this node - you want the regular Load FLOAT Models (Opt) + FLOAT Process (Opt) combo, which loads everything in one shot and is dramatically simpler. The VA nodes exist for people who want to pull the pipeline apart: swap in a custom audio encoder, tweak one subnetwork, or chase down that last bit of VRAM. The README is explicit that VA exists for investigating options, not for casual use. Use it when you mean it.

    What it does

    The node loads decoder.safetensors (or synthesis.safetensors) from models/float/motion_autoencoder/, then does something clever: it infers the architecture from the weights instead of asking you for every number. It reads the tensor shapes to figure out the image size, the style dimension, and the motion dimension, and hands those back to you as inferred_size, inferred_style_dim, and inferred_motion_dim. That's genuinely useful - those ints let downstream nodes validate their wiring, and they're the values you'd otherwise have to dig out of the paper.

    But not everything is inferable from weights, so two hyperparameters are on you:

    • channel_multiplier - an architectural setting for the module's channel counts. Default 1. It must match the value the weights were trained with, or you'll get shape mismatches downstream.
    • blur_kernel_str - the blur kernel used by the upsampling layers, entered as a Python list string like [1, 3, 3, 1]. Same rule: match the training config. The tooltip is literally "Should match the value used to train the loaded weights" - this node trusts you to know what you're doing.

    Plus the usual target_device and cudnn_benchmark (keep benchmark off unless you've profiled it on; it makes the first run slow).

    Outputs and wiring

    The important output is float_synthesis (a FLOAT_SYNTHESIS_MODEL). It feeds two places in the VA graph: FloatGetIdentityReferenceVA, which derives the identity vector from your reference photo, and ApplyFloatSynthesis, which takes that identity plus the motion latents and actually renders the frames. In the README's workflow map: image path → Encoder → Synthesis; sampling path → Synthesis. The three inferred ints are there for validation and for building the graph programmatically.

    Where the weights come from

    You don't have to hunt down a decoder.safetensors file yourself. If it's missing, the loader checks for the unified FLOAT.safetensors and, if that's present, extracts the decoder part from it automatically. Only if neither exists does it download the component directly. So the VA workflow still bootstraps off that same 2.4 GB download - it just splits it up after.

    If you do want to manage parts by hand, the README's "Very Advanced" manual-download section points at the individual FLOAT parts on HuggingFace (set-soft/float_advanced), copied into models/float/.

    Troubleshooting

    • Missing weights error ("Synthesis weights file not found") means the extraction from the unified model failed or the folder's empty - check that FLOAT.safetensors is actually in models/float.
    • Shape mismatches downstream almost always trace back to channel_multiplier or blur_kernel_str not matching the checkpoint. Revert to defaults first.
    • The VA nodes' inputs and outputs are still moving - the README warns they "might change." Don't build a permanent workflow on them yet.

    One line to remember: this node loads FLOAT's renderer. The brain that makes the face move is a different loader - that's Load FLOAT FMT Model (VA).

    CategoryFLOAT/Very Advanced/Loaders

    Inputs (5)

    NameTypeDefaultDescription
    synthesis_fileCOMBOThe .safetensors file containing the pre-trained weights for the Synthesis (Decoder).
    target_deviceCOMBOcpuThe device (CPU or CUDA) where the Encoder will run during inference.
    channel_multiplierINT11–8Architectural hyperparameter for the Synthesis module's channel counts. Must match the value used to train the loaded weights.
    blur_kernel_strSTRING[1, 3, 3, 1]Architectural hyperparameter defining the blur kernel for upsampling layers, as a Python list string (e.g., '[1,3,3,1]'). Should match the value used to train the loaded weights.
    cudnn_benchmarkBOOLEANfalseEnable or disable cuDNN benchmarking for this model's operations. Can improve speed.

    Outputs (4)

    NameTypeDescription
    float_synthesisFLOAT_SYNTHESIS_MODEL
    inferred_sizeINT
    inferred_style_dimINT
    inferred_motion_dimINT