Nodes/ComfyUI-FLOAT_Optimized/FLOAT Encode Image to Latents (Ad)
ComfyUI Node

FLOAT Encode Image to Latents (Ad)

Where the face becomes a puppet

By set-soft·Created about a year ago·Updated 12 months ago· 33
FLOAT Encode Image to Latents (Ad)
  • ref_image
  • float_pipe
  • appearance_pipe (Ws→r)
  • r_s_lambda_latent
  • float_pipe

FLOAT's whole trick is splitting your reference photo into two things: what the person looks like and what pose they happen to be in. FLOAT Encode Image to Latents (Ad) is the first half of that split. You feed it a reference image plus the float_pipe, and it runs the image through the motion autoencoder's encoder to pull out the appearance latent and multi-scale feature maps (bundled into the appearance_pipe) and a separate motion-control vector (r_s_lambda_latent).

Think of it as building the puppet. The appearance pipe is the puppet's body - the person's identity, lighting, background - and r_s_lambda_latent is where the puppet's current pose is noted down. Later, a sampler will generate a sequence of motion that gets combined with that appearance to produce frames.

Inputs and outputs

Only two inputs:

  • ref_image - your reference photo, correctly sized for the encoder. This is the "Advanced" node, so it assumes you've already aligned and squared it - in practice you'll want a Face Align for FLOAT node upstream, or a manually prepared 512×512 square headshot. There's no internal face detection here.
  • float_pipe - from Load FLOAT Models (Opt).

Outputs:

  • appearance_pipe (Ws→r) - the bundled s_r appearance latent plus multi-resolution feature maps. This gets threaded to FloatDecodeLatentsToImages at the end (and, if you're curious, the arrow notation Ws→r is FLOAT's way of saying "the static appearance flows into the rendered frame").
  • r_s_lambda_latent - the motion-control parameters for this particular photo. This is not the final identity reference; it still needs one more hop through FLOAT Get Identity Reference to become r_s, the motion-space identity vector the sampler actually uses.
  • float_pipe - passthrough, so the pipe keeps flowing down the graph.

That "one more hop" is the classic beginner stumble: r_s_lambda_latent looks like the thing you'd feed the sampler, and it isn't. Follow the documented chain - encode image → get identity reference → then sample.

Why you'd use this instead of the one-button node

The "Opt" node runs this whole encode-audio-predict-emotion-sample-decode gauntlet in one go. This Advanced line exists so you can actually see and rewire the pieces: you can run several reference images through the encoder and reuse the same audio path, or cache the appearance and only vary the driving signal. It's also where the face-margin and RGBA settings from Float Advanced Options take effect, since alignment happens upstream.

Install is the pack install: ComfyUI Manager (search "ComfyUI-FLOAT_Optimized"), or

cd ComfyUI/custom_nodes
git clone https://github.com/set-soft/ComfyUI-FLOAT_Optimized
pip install -r requirements.txt

then restart, and let the 2.4 GiB FLOAT.safetensors download to models/float on the first load. The only real gotcha here is the one baked into FLOAT generally: it's CC BY-NC-SA 4.0, so it's not for commercial work, and the motion you get out is only as good as the square, clean, well-margined headshot you put in.

CategoryFLOAT/Advanced

Inputs (2)

NameTypeDefaultDescription
ref_imageIMAGE
float_pipeFLOAT_PIPE

Outputs (3)

NameTypeDescription
appearance_pipe (Ws→r)FLOAT_APPEARANCE_PIPE
r_s_lambda_latentTORCH_TENSOR
float_pipeFLOAT_PIPE