AP Warp Image + Mask by RAFT Flow

Optical flow is "who moved where between this frame and that one," and the trickiest part of using it in ComfyUI is keeping the thing that moved - and its mask - moving together. AP Warp Image + Mask by RAFT Flow does exactly that: it takes a precomputed flow field and uses it to warp an image and a mask in lockstep. It's the "apply" half of the optical-flow equation in Adam Polczynski's AP_OpticalFlow pack, and it's the node you reach for when you want a frame (or a region of it) to follow motion you measured, not motion you guessed.

The name is worth taking literally: this node warps, it doesn't composite. Feed it flow_data from AP Get RAFT Optical Flow (or AP Load Optical Flow if you cached the flow to disk), and you get a warped image, a warped mask, and a valid_mask telling you where the warped pixels actually came from the source. Whether you then blend the warp back onto the original is a separate step.

How it works

The warp is a grid_sample under the hood: it builds a normalized coordinate grid, adds the flow field to it (scaled by strength), and samples the image at those displaced positions. The mask is warped with the same flow but zeros padding, so it doesn't smear paint at the borders. The valid_mask output is the safety net - it marks every pixel whose source coordinates fall inside the original image. Pixels the flow pushes off the edge are simply missing, and valid_mask is how you know which ones those are.

The inputs that matter

• images, mask, flow_data - the frame, its region mask, and the flow. The flow is auto-resized to match the image, so mismatched resolutions won't crash you.
• flow_direction (ab / ba) - if motion comes out reversed, flip this or tick invert_flow. The README calls this out directly, and it's the most common "why is it wrong" cause.
• strength (default 1) - scales the flow; go negative to reverse direction, go under 1 to ease off the motion.
• batch_mode + current_frame_index - for loop pipelines. by_index expects a batch of one image and picks the flow entry for the current frame index; auto does the same when an index is connected. flow_skip / frames_skip offset the flow timeline relative to the frame timeline.

Outputs

warped_image (IMAGE) and warped_mask (MASK) are the moved content. valid_mask (MASK) is the "this pixel is real data" map. The canonical next stop is AP Flow Composite, which blends the warp back using the valid mask - or just skip the manual chain and use AP Warp Masked Composite Blend (Occlusion), the pack's one-node version that does warp + occlusion + composite in a single step.

Install

ComfyUI Manager, search "AP Optical Flow", install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/adampolczynski/ComfyUI_AP_OpticalFlow
pip install -r ComfyUI_AP_OpticalFlow/requirements.txt

The only requirement is torchvision>=0.15; RAFT's pretrained weights download automatically the first time you run AP Get RAFT Optical Flow. Troubleshooting is mostly the two classics: motion backwards → swap flow_direction, and smeared/black edges → that's valid_mask territory, so composite through it rather than blaming the node. If you're getting batch errors in a loop, remember by_index wants a batch of exactly one image.