Nodes/ComfyUI-FLOAT_Optimized/Sample Motion Sequence RD (VA)
ComfyUI Node

Sample Motion Sequence RD (VA)

The whole flow-matching engine in one node

By set-soft·Created about a year ago·Updated 12 months ago· 33
Sample Motion Sequence RD (VA)
  • r_s_latent
  • wa_latent
  • we_latent
  • float_fmt_model
  • r_d_latents (Wr→D)
  • float_fmt_model_out
audio_num_frames
a_cfg_scale2.0
r_cfg_scale1.0
e_cfg_scale1.0
include_r_cfgfalse
nfe10
torchdiffeq_ode_methodeuler
ode_atol0.0000
ode_rtol0.0000
audio_dropout_prob0.10
ref_dropout_prob0.10
emotion_dropout_prob0.10
fix_noise_seedtrue
seed15

Every branch of the Very Advanced graph - image, audio, emotion - converges on this node. Sample Motion Sequence RD (VA) is the FLOAT sampler, the node that takes the identity latent (r_s), the audio conditioning (wa), and the emotion conditioning (we), and generates the entire driven-motion sequence (r_d) that becomes every frame of your talking head. It's the biggest node in the pack, and for good reason: it exposes the Flow Matching Transformer's own ODE solver, CFG scales, and noise handling directly, with no pipe in the way.

The mechanism is flow matching, and the mental model is worth building once because it makes the parameter list legible. FLOAT learns a velocity field - "given where this motion is and these conditions, where should it go next?" - and the sampler starts from random noise and integrates along that field from time 0 to 1. The integration is done by a torchdiffeq ODE solver, and every knob here is either "how carefully to integrate" or "how hard to push a condition." No diffusion steps, no Karras, no denoise strength - that's the SD muscle memory to unlearn.

The inputs that matter

  • float_fmt_model - the loaded Flow Matching Transformer from Load FLOAT FMT Model. This is the VA-graph difference: the model is right there, swappable.
  • r_s_latent, wa_latent, we_latent - from Float Get Identity Reference VA, Float Apply Audio Projection, and the emotion node of your choice.
  • audio_num_frames - link-only; the sampler must generate exactly this many frames so the video matches the audio.
  • a_cfg_scale (default 2) / e_cfg_scale (default 1) - audio and emotion guidance. These run the model with a condition dropped and extrapolate toward "follow it harder." Lip-sync tightness and emotional intensity, respectively.
  • r_cfg_scale (default 1) - reference guidance, and a subtle trap: the tooltip admits it's currently unused in the default FMT implementation. It only does anything if you flip include_r_cfg (false, experimental), which adds a fourth forward pass and makes r_cfg_scale live.
  • nfe (default 10, up to 1000) - Number of Function Evaluations. Your quality/speed dial. 10 is the shipped default; more = finer integration, slower.
  • torchdiffeq_ode_method - euler, midpoint, rk4, heun2, heun3. Euler is fastest and the default; rk4 is the accuracy pick.
  • ode_atol / ode_rtol (both 1e-05) - solver tolerances; leave them alone unless you're debugging.
  • audio/ref/emotion_dropout_prob (all 0.1) - dropout probabilities for CFG. Higher = more variation, and they're the mechanism behind the guidance scales.
  • fix_noise_seed (default true) + seed - reproducible noise. Keep fix_noise_seed on and change the seed for a new performance; the ODE is deterministic given the noise.

Outputs: r_d_latents (Wr→D) (into Apply Float Synthesis) and the FMT model passthrough.

Where people get burned

The classic misses: feeding r_s_lambda_latent instead of r_s_latent (the identity hop is a separate node, don't skip it), expecting r_cfg_scale to do something before enabling include_r_cfg, and bumping nfe to 100 and wondering why it's slow. Start at defaults, tune a_cfg_scale and e_cfg_scale first - they have the biggest visible effect per unit of effort.

Install is the pack standard: Manager search "ComfyUI-FLOAT_Optimized", or clone into custom_nodes + pip install -r requirements.txt, restart. The FMT weights extract from the unified FLOAT.safetensors or download as fmt.safetensors. And the two standing VA-graph caveats: interfaces "might change," and FLOAT is CC BY-NC-SA 4.0 - non-commercial.

CategoryFLOAT/Very Advanced

Inputs (18)

NameTypeDefaultDescription
r_s_latentTORCH_TENSORThe reference identity latent (wr), derived from the source image. Wr→s
wa_latentTORCH_TENSORThe audio conditioning latent (wa), derived from the audio features.
audio_num_framesINTTotal number of frames to generate, determined by the audio length and target FPS.
we_latentTORCH_TENSORThe emotion conditioning latent (we), derived from emotion prediction or specification.
float_fmt_modelFLOAT_FMT_MODELThe loaded FlowMatchingTransformer model from a loader node.
a_cfg_scaleFLOAT2.00–10Audio Guidance Scale. Higher values make the motion follow the audio more strictly.
r_cfg_scaleFLOAT1.00–10Reference Identity Guidance Scale. (Note: Currently unused in the default FMT implementation).
e_cfg_scaleFLOAT1.00–10Emotion Guidance Scale. Higher values make the motion express the target emotion more strongly.
include_r_cfgBOOLEANfalseExperimental! Try to include some control over the reference weight. When enabled the `r_cfg_scale` is used.
nfeINT101–1000Number of Function Evaluations for the ODE solver. Higher values increase quality and generation time.
torchdiffeq_ode_methodCOMBOeulerThe specific fixed-step numerical integration method for the ODE solver.
ode_atolFLOAT0.00001e-9–0.1Absolute tolerance for the ODE solver. Controls precision.
ode_rtolFLOAT0.00001e-9–0.1Relative tolerance for the ODE solver. Controls precision.
audio_dropout_probFLOAT0.100–1Dropout probability for the audio condition during sampling. Set > 0 for variation.
ref_dropout_probFLOAT0.100–1Dropout probability for the reference condition during sampling. Set > 0 for variation.
emotion_dropout_probFLOAT0.100–1Dropout probability for the emotion condition during sampling. Set > 0 for variation.
fix_noise_seedBOOLEANtrueIf true, the 'seed' input will be used to generate reproducible noise. If false, behavior depends on the seed value.
seedINT150–18446744073709550000The seed for the random noise generator used by the ODE sampler.

Outputs (2)

NameTypeDescription
r_d_latents (Wr→D)TORCH_TENSOR
float_fmt_model_outFLOAT_FMT_MODEL