vlo Time-to-Move (TTM)

The most frustrating thing about video generation is that you can't tell the model how to move. You get motion by praying. vlo Time-to-Move (TTM) is the end of that - it patches a video model so a generation follows the exact motion of a reference clip while generating new content for the pixels. Drag a clip onto the timeline, tell it where the subject is, and the model re-does the shot with whatever you want in it, keeping the choreography.

It's the "cut-and-drag" trick, and it's a headline feature of vlo, the video editor this pack serves - the editor even animates its UI input to drive the reference. The underlying framework, Time-to-Move, is real and community-verified (it's the reason the Wan 2.2 motion-transfer workflows exist). This node is a clean ComfyUI-native port, and the pack marks it experimental, so treat it as a sharp tool.

How it works

The core idea is subtle and worth getting right: the motion lives in the starting latent, not in any conditioning. TTM encodes your reference clip into latent space and uses it as the sampler's initial latent. Partially noised, that latent still carries the motion structure - the model denoises from there, so the movement comes along.

The node implements this by patching the model with an OUTER_SAMPLE wrapper that does two things: it drops the sampler's first start_step sigmas (so the reference is seeded at a specific noise level instead of pure noise), and it drives ComfyUI's own inpaint path to hold the masked region to the reference for the opening steps before letting it denoise freely. Because it rides Comfy's native inpaint machinery, it works with any sampler. Patch only the model of the sampler that starts the schedule, and leave that sampler's start_at_step at 0.

Inputs that matter

• model - the video model to patch (output is the patched model).
• reference_latents - your encoded reference clip. Must match the resolution and frame count being sampled, exactly - the node errors otherwise.
• mask - white holds the region to the reference, black is free for the model. For a moving subject that means a white background and a black hole over the subject.
• start_step (default 1) - which step's noise level seeds the reference. Higher binds tighter to the reference (less noise on it) but eats a step and gives the model less room to clean up paste artifacts. 0 is a no-op: sigma is 1.0 there and the reference washes out entirely.
• end_step (default 2) - exclusive; the step where the region stops being held. At or below start_step, it seeds the init and never holds - useful as a pure motion-seed.

Where it bites

• Shape mismatch is fatal. Reference must match the sampled latent's resolution and frame count, or you get a hard error.
• Noise-free samplers skip it. A one-step or distilled sampler that adds no noise logs a warning and runs without TTM - there's nothing to seed into. Patch the sampler that starts the schedule, not a downstream one.
• It replaces any noise_mask already on the sampled latent; TTM drives the mask itself.
• Mask alignment. The node strides your mask frames onto the latent grid using the model's temporal downscale (4 for LTX-style VAEs). Feed a mask whose frame count matches the reference video; if it doesn't stride cleanly, it falls back to nearest-frame resampling and can misalign - and a misaligned mask is how a moving subject leaks into the held background.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/PxTicks/ComfyUI-vlo.git

Restart ComfyUI (or use ComfyUI Manager, "ComfyUI-vlo"). No models, no extra deps - it patches a model at runtime, so you just need the video checkpoint you're already using.