Nodes/ComfyUI Stat-Imp Nodes/Compute Optical Flow
ComfyUI Node

Compute Optical Flow

Optical flow for smarter tweens (and when to skip it)

By Statistical-Impossibility·Created 4 months ago·Updated 3 months ago· 0
Compute Optical Flow
  • image_from
  • image_to
  • flow
◄method▾►
◄deviceAUTO►

The Cadence node dissolves its in-between frames in place, which is fine for gentle motion and slightly smeary for fast moves. Feed it a flow field - the per-pixel displacement between two frames - and it can warp the two image lineages into alignment before dissolving, so motion doesn't smear. That's the entire job of Compute Optical Flow: estimate the flow, hand it to Cadence's flow input.

It's deliberately a separate node from Cadence rather than a built-in option, and that's a good design call: the flow backend is swappable, so you can pick the estimator that fits your GPU, your frame quality, and your patience.

How it works

You give it two frames, it gives you a FLOW tensor - (1, H, W, 2), absolute pixel displacement mapping image_from → image_to, sized to image_to. (If the two frames differ in resolution, image_from gets resized to match first; on the bootstrap frames where the lineages are different sizes, that's a meaningful convenience.) Cadence then uses that flow to align the lineages before the cross-dissolve on tween frames.

The method dropdown is where the real choice lives:

  • None - zero flow, identity field. This is Cadence's plain v1 dissolve with the flow input effectively disconnected. No dependency, and if your frames are noisy it's often the right answer.
  • DIS Medium / DIS Fine - OpenCV's DIS optical flow, CPU. These are the stock vanilla-Deforum presets, ported faithfully (DIS Fine is the custom fine-scale variant with 192 gradient-descent iterations).
  • Farneback - the other classic OpenCV method, also CPU.
  • RAFT - the deep-learning estimator via torchvision. GPU or CPU (device dropdown: AUTO / cuda / cpu, and only RAFT pays attention to it). Weights auto-download on first use.

Dependencies

This is the one node in the pack with real extra requirements, and they're optional per backend: DIS Medium / DIS Fine / Farneback use OpenCV, which is almost always already bundled with ComfyUI. RAFT needs torchvision plus a one-time weight download on first run. Everything else in the pack is pure torch. If you don't want to think about it, leave method at None or one of the DIS options and skip the torchvision install entirely.

When to actually use flow

The honest guidance from the pack: flow-based morphing shines on clean, high-quality frames. On heavily distilled / low-step models (SDXL Turbo, Lightning, Hyper - the exact models the shipped example workflows target) the per-frame texture noise makes estimated flow noisy, and noisy flow warps your tweens in wrong directions. In that regime None (plain dissolve) or Farneback/DIS beats RAFT - the deep model amplifies the noise rather than fixing it. Try None first, add flow only if the tweens look smeary on fast motion, and if a flow-warped tween looks wobbly, back flow_factor down on the Cadence node before you change the whole backend.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/Statistical-Impossibility/comfyui-Stat-Imp-nodes

Restart and hard-refresh, and it's under Stat-Imp / Deforum. The node works standalone (just give it two images and read the flow), but its intended home is inside a Deforum loop with Cadence - which needs the pack's fork of the harness (https://github.com/Statistical-Impossibility/deforum-comfy-nodes) to carry the extra loop state. First RAFT run will pause to fetch weights, so don't panic when it takes a moment; the OpenCV backends never do.

CategoryStat-Imp/Deforum

Inputs (4)

NameTypeDefaultDescription
image_fromIMAGE—
image_toIMAGE—
methodCOMBO5 options: None, RAFT, DIS Medium, DIS Fine, Farneback
deviceCOMBOAUTO3 options: AUTO, cuda, cpu

Outputs (1)

NameTypeDescription
flowFLOW—