(deforum) Apply Flow

Most "make a photo move" tricks fake motion with a camera transform - zoom, pan, rotate. (deforum) Apply Flow does something different: it computes real optical flow between two frames and warps an image along that flow. The result is motion that follows the actual movement in a scene, which is exactly what you want when you're blending a Deforum render with a live video - the "hybrid" workflows that made Deforum famous.

How it works

The node keeps a two-image memory. Feed it image (the frame you want to warp) and flow_image (the reference whose motion you want to borrow), and it computes the optical flow between the last two images it's seen using your chosen method, then displaces the image pixels along that flow field. flow_factor (0–1, default 0.8) scales how far the pixels actually travel - it's the motion-strength dial.

The flow methods are worth knowing:

• RAFT - the deep-learning flow model (loads weights on first use). Best quality, slowest.
• DIS Medium / DIS Fine - the DIS (Deep Image Smoothing) estimator at two quality settings. Good middle ground, much faster than RAFT.
• Farneback - classic OpenCV dense flow. Fastest and no model download, but the coarsest on large motion.

The inputs and outputs that matter

• image - the frame to warp.
• flow_image - the reference image whose motion drives the warp. If you leave it unset, the node uses image itself and the flow comes from the node's internal pair history.
• flow_method - RAFT / DIS Medium / DIS Fine / Farneback.
• flow_factor (FLOAT, 0–1, default 0.8) - how far pixels move along the flow.
• deforum_frame_data (optional) - when its reset flag is set, the node clears its internal image cache (so a new animation doesn't leak motion from the previous run).

Output:

• IMAGE - the flow-warped frame.

Installing this pack

Part of Deforum Nodes by XmYx. ComfyUI Manager: search Deforum Nodes, or:

cd ComfyUI/custom_nodes
git clone https://github.com/XmYx/deforum-comfy-nodes

Restart and let install.py install the deforum-studio backend from GitHub plus the scientific-Python stack (numpy < 2.0; Python 3.10 required). Choosing RAFT triggers a model download on first run; the DIS and Farneback methods need nothing extra.

Common issues

• First frame comes back unchanged. Expected - the node needs two frames in its cache before it can compute flow. The first image is just stored.
• Rendering with RAFT is slow. It's a full deep model. For previews use Farneback or DIS; switch to RAFT for the final pass.
• Motion leaks between animations. If the previous run's flow is still influencing frames, the deforum_frame_data reset isn't reaching this node. Wire it so the Iterator's reset reaches here, or the cache isn't cleared.
• Warps look stretched at the edges. Pixels have to come from somewhere. flow_factor below 1.0 leaves some original structure, which usually looks better than full displacement.