ComfyUI Node

Encode WanVideo

The node that turns your 'motion signal' into Wan latents

By GiusTex·Created 9 months ago·Updated 5 months ago· 8
Encode WanVideo
  • vae
  • image
  • mask
  • reference_latents
enable_vae_tilingfalse
tile_x272
tile_y272
tile_stride_x144
tile_stride_y128
noise_aug_strength0.0
latent_strength1.0

TimeToMove (TTM) is the Wan 2.2 trick where you don't describe motion with words - you show it. You cut a subject out of a still (a car, a toy, a character), nudge the cutout through the scene in any video editor to make a crude "motion signal" clip, and Wan re-renders it as a smooth, photoreal video that tracks that movement. The "Time-to-Move + Wan 2.2 Test" demo that lit up r/StableDiffusion (5100+ upvotes) runs on exactly this idea, and this node is the first stop in the pipeline.

Encode WanVideo is a direct port of the same node from Kijai's WanVideoWrapper, and it does one job: it takes your driving/reference video and encodes it into the latent space of your Wan VAE. Nothing downstream in a TTM workflow works until this node has produced those reference latents.

How it works

It feeds your image tensor (the frames of your driving video) through the Wan VAE into latent space, exactly like the encode step in any Wan I2V workflow - just pointed at a whole video instead of a single frame. A few details worth knowing:

  • If the image has 4 channels (RGBA), it silently drops the alpha - handy if your editor exports with transparency.
  • noise_aug_strength adds Gaussian noise to the reference before encoding, at a strength you pick. This is a Leapfusion I2V trick: a little noise can add motion and sharper results.
  • latent_strength is a multiplier on the encoded latents. Lower it toward 0 and the reference influences the output less, which is how you let the model move more freely.
  • The output latents carry a noise_mask when you pass a mask in, so downstream nodes know what region is "real."

Inputs and outputs

Required: vae, image, plus the VAE tiling quartet. Optional: noise_aug_strength, latent_strength, mask. Output: reference_latents (a LATENT).

The ones you'll actually touch:

  • image - feed your driving video frames here (from a LoadVideo/VideoLoad node in a TTM workflow).
  • enable_vae_tiling - leave off unless you're VRAM-starved. It drastically cuts memory use but the author's own tooltip admits tiling "may introduce seams"; the tile_x/y and tile_stride_x/y sizes below it only matter if tiling is on.
  • noise_aug_strength / latent_strength - ignore these unless you're doing Leapfusion I2V. Defaults (0 and 1) are right for stock TTM.

reference_latents wires straight into TTM Latent Add's reference_latents input. That's its only job.

Installing

The pack has no third-party dependencies - no requirements.txt, nothing to pip install - so it's the easy install:

cd ComfyUI/custom_nodes
git clone https://github.com/GiusTex/ComfyUI-Wan-TimeToMove.git

Restart ComfyUI, and the three nodes show up under "Wan22 TimeToMove" in the node menu (ComfyUI Manager finds it too, if you search the pack title). The pack ships no model files; you bring your own Wan 2.2 I2V model, its VAE (wan_2.1_vae still works for 2.2), and the umt5 text encoder - the standard Wan stack. The repo's example workflow also pulls in the CFGFloatListScheduler (moved to GiusTex's ComfyUI-MoreEfficientSamplers) and Kijai's WanVideoWrapper for the first/last-frame node, but the TTM nodes themselves don't need any of it.

Where people get burned

  • The reference must actually be a motion signal. Encode the wrong thing (your still first frame, an unrelated clip) and TTM either does nothing or fights the prompt. The driving video is the whole point of this technique.
  • Tiling seams. If you flip on tiling to fit VRAM, check a few frames for grid seams before committing to a long render.
  • Wan's VAE compresses time ~4x. A 97-frame driving clip becomes ~25 latent frames. That's normal; don't panic when the encoded shape looks short.
CategoryWan22 TimeToMove

Inputs (10)

NameTypeDefaultDescription
vaeVAE
imageIMAGE
enable_vae_tilingBOOLEANfalseDrastically reduces memory use but may introduce seams
tile_xINT27264–2048Tile size in pixels, smaller values use less VRAM, may introduce more seams
tile_yINT27264–2048Tile size in pixels, smaller values use less VRAM, may introduce more seams
tile_stride_xINT14432–2048Tile stride in pixels, smaller values use less VRAM, may introduce more seams
tile_stride_yINT12832–2048Tile stride in pixels, smaller values use less VRAM, may introduce more seams
noise_aug_strengthoptFLOAT0.00–10Strength of noise augmentation, helpful for leapfusion I2V where some noise can add motion and give sharper results
latent_strengthoptFLOAT1.00–10Additional latent multiplier, helpful for leapfusion I2V where lower values allow for more motion
maskoptMASK

Outputs (1)

NameTypeDescription
reference_latentsLATENT