ExtractOpticalFlow

This is the heart of ComfyWarp. Give it two frames of your source video - the current one and the previous one - and it figures out, per pixel, where the previous frame's content has moved to in the current frame. That motion field (optical flow) is what lets the pipeline warp the stylized version of the previous frame forward so your style travels with the camera instead of smearing or resetting every frame. It's the same trick WarpFusion popularized back in the A1111 days, sitting somewhere between Deforum (which cheats with zoom/perspective transforms) and AnimateDiff (which learns motion). Deforum can't represent real parallax; this can, because it's derived from the actual video.

How it actually works

Under the hood it runs RAFT, the classic torchvision optical-flow model, in fp16. The weights (the C_T_SKHT_V1 RAFT-Large weights) download automatically the first time the node runs - no manual model file to hunt down. If you have CUDA it uses your GPU; otherwise it silently falls back to CPU, and I'll warn you now: CPU RAFT on 30fps footage will make you question your life choices. num_flow_updates (default 20) is how many iterative refinement passes RAFT does; more passes = more accurate flow, slower. The node clamps you between 5 and 100, and in practice you rarely need to touch it unless flow is clearly breaking down on fast motion.

The interesting part is the masks it hands back alongside the flow:

• Motion edge mask - edges in the flow field, where motion changes sharply. These are your worst ghosting zones.
• Occlusion mask - pixels in the current frame that have no counterpart in the previous frame, because they were covered up (or uncovered). Warping the previous frame can't produce these, so you must not trust warped content there.
• Border mask - pixels whose motion would push the previous frame's content off the edge of the frame.

The Flow preview image is a color-coded visualization of the motion, great for sanity-checking that the flow actually matches what the camera did - if the preview looks like static noise, something upstream is wrong.

Wiring it up

current_frame and previous_frame are both plain IMAGEs - usually the outputs of LoadFramePairFromDataset, which conveniently hands you both plus the frame number. All three masks and the preview come out as separate ports, and you'd normally feed the masks into MixConsistencyMaps to turn them into one useful compositing mask. The Flow output (BACKWARD_FLOW) plugs into WarpFrame to actually warp your stylized previous frame forward.

Two real-world tips. First, both input frames must be the same resolution or the flow math gets confused - that's what ResizeToFit is for. Second, there's a subtle behavior worth knowing: if the whole scene is basically static (max flow magnitude below a tiny threshold), the node zeroes out the flow to avoid producing noisy, meaningless masks. That's intentional, not a bug - it keeps a locked-off shot from generating fake motion trails.

It's the node you'll reach for constantly once you're inside ComfyWarp, and it's also the one that teaches you the most about why the pipeline works. If the flow preview looks right, everything downstream has a chance.