Compute optical flow

If you've ever run a vid2vid workflow and watched the subject swim, morph, or slide between frames, the problem is that every img2img pass throws noise back into the image. The frame you generated doesn't know where the source video moved, so it guesses, and the guess gets worse every frame. This node is the thing that stops the guessing: it computes optical flow between two images - usually adjacent frames - and hands you the exact motion field. It's the first of three nodes in the seanlynch/comfyui-optical-flow pack, and everything else plugs into what it produces.

Optical flow, in one line: for every pixel in image A, where did that pixel end up in image B? The output is a two-channel displacement map, one channel per axis, telling you the x and y offset at each pixel. That map is your motion ground truth for the shot - a car driving left produces a wall of arrows pointing left, a camera zoom produces arrows radiating outward.

How it works

The node takes two IMAGE tensors, prev and current, converts them to grayscale, and runs a dense optical flow algorithm from OpenCV on the CPU. No GPU involved, no neural network - this is classical computer vision, and it's fast.

Your one real decision is method, a dropdown with three options:

• DIS Medium - Dense Inverse Search with the medium preset. The sensible default: accurate, quick, and rarely needs you to think about it.
• DIS Fine - the same algorithm with the fine preset (192 gradient-descent iterations at the finest scale). Better on subtle, small-scale motion; costs noticeably more CPU time.
• Farneback - the classic calcOpticalFlowFarneback, the old workhorse. It's the one to reach for if DIS gives you weird artifacts on your footage, and the one you'll usually skip otherwise - it's chunkier and slower for the same quality.

All three are OpenCV, so the results you get are deterministic and cheap. On a 1080p frame, DIS Medium typically finishes in well under a second; DIS Fine will make you wait a bit.

One nice touch you might miss: the node is batch-aware. Feed it lists of frames (say, from a Load Image Batch node) and it pairs them up position-by-position, returning one flow per pair in a single pass. If prev and current have different frame counts, it just processes up to the shorter list.

Inputs and outputs

• prev - the earlier frame (or frames).
• current - the later frame. Both must be the same dimensions, or the OpenCV call blows up; the README is explicit about this, and it's the #1 way people trip this node.
• method - DIS Medium, DIS Fine, or Farneback.

The single output is OPTICAL_FLOW. Note that this isn't a native ComfyUI type - it's a custom type string this pack registered, so it's an opaque blob as far as ComfyUI is concerned. It plugs into the pack's Apply optical flow and Visualize optical flow nodes, and nowhere else. Don't try to wire it into a sampler or VAE; nothing else will accept it.

Installing it

Open ComfyUI Manager, search for ComfyUI Optical Flow, and hit install. Or do it by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/seanlynch/comfyui-optical-flow

Then restart ComfyUI. That's the whole install - there's no requirements.txt and no model download. The pack only needs OpenCV, NumPy, and PyTorch, all of which ComfyUI ships with. It's a refreshingly light dependency for a video-adjacent node.

Gotchas

• Dimension mismatch is a hard error, and it's the most common complaint. Resize both inputs to the same resolution before computing.
• It's CPU-only. If you're computing flow on a whole video's worth of pairs with DIS Fine, that's real time - but it's still far cheaper than a single diffusion pass, so it won't be your bottleneck.
• The flow you get is only as good as the frames you feed it. Compute flow between two unrelated frames and you'll get nonsense arrows - and you'll catch that by visualizing the flow before you trust it.

That's where the other two nodes come in: Visualize optical flow turns the invisible map into something you can look at, and Apply optical flow does the actual warping that keeps your frames consistent. This node is the start of that chain, and it's the only one with real settings to fiddle with.