Sample Trajectories

Every FLATTEN workflow starts here

FLATTEN's whole trick is that attention follows motion. Before any of that can happen, something has to read the motion out of your video - and that's this node. Sample Trajectories runs optical flow over your input frames and packages it into a TRAJECTORY that every other node in the pack (the loader, the unsampler, the KSampler, the attention node, the flow-noise node) consumes. No trajectories, no FLATTEN.

The optical flow itself comes from RAFT-large, and here's the nice part: it's the version bundled inside torchvision (raft_large with the default pretrained weights). That's why this pack needs zero extra Python packages and no separate model download - the "model" is sitting inside your existing torchvision install, downloaded on first use like any other pretrained torchvision weight. It runs on your ComfyUI device and shows progress in the usual progress bar.

How it works

It splits your frames into windows of context_length frames with context_overlap overlap, runs RAFT over each window, and stores per-window trajectories plus their visibility masks, keyed by the starting frame index. The windowing exists because optical flow at full video length eats VRAM - and, conveniently, it matches the batching windows that AnimateDiff-Evolved uses. The trajectory dict also records the input image's height and width, because downstream nodes hardcode an 8× downscale to the latent grid and need those dimensions to line up.

The three inputs

• images - your input frames, an IMAGE batch (load them with Video Helper Suite if they came from a video file).
• context_length - frames per window, default 20, capped at 40. This defines how much FLATTEN processes at once; dropping it cuts VRAM usage.
• context_overlap - overlap between windows, default 10.

The single output is TRAJECTORY, which you wire into the rest of the pack.

The traps

Two things bite people, both in the README. First: trajectories are created for the dimensions of the input image and must match the latent size FLATTEN processes. If you sample trajectories at one resolution and feed a differently-sized latent, the flow lookup lands in the wrong place. Second: batching only works with AnimateDiff-Evolved's Standard Static context, and the context_length/context_overlap here must match the values you give AnimateDiff's Evolved Sampling context - mismatch them and your windows stop aligning. Context Views aren't supported, and scheduled prompting is a known no-go too.

Installing

Search ComfyUI-FLATTEN in ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/logtd/ComfyUI-FLATTEN

Restart ComfyUI. Nothing extra to pip-install - the pack's requirements.txt is empty and RAFT comes with torchvision. For real batching you'll additionally want Kosinkadik's ComfyUI-AnimateDiff-Evolved and, to pull frames out of a video, ComfyUI-VideoHelperSuite.