OpenCV calcOpticalFlowFarneback_0

Optical flow answers "where did every pixel move between frame A and frame B?" OpenCV calcOpticalFlowFarneback_0 computes dense flow - a full motion field, one displacement vector per pixel - using the classic Farneback algorithm. In a ComfyUI context this is the kind of thing you'd reach for if you're doing motion analysis on video frames: feeding consecutive frames from a video loader to see where content moved, driving motion-based blending, or experimenting with frame-interpolation-adjacent tricks. It's a real computer-vision workhorse, not a toy - and it's also one of the pack's least beginner-friendly nodes, for reasons that are mostly the generator's fault, not yours.

The algorithm itself is solid and well-tested: it approximates local neighborhoods with quadratic polynomials, matches them between frames, and refines across an image pyramid. The output is a two-channel array where channel 0 is horizontal displacement and channel 1 is vertical. Fast enough for offline frame analysis, and it's been production-hardened for twenty years.

Inputs that matter

• prev / next - the two NPARRAY frames, grayscale uint8 required. Feed it RGB and OpenCV throws (-215:Assertion failed) img.type() == CV_8UC1 - the README's own troubleshooting entry. Convert with cvtColor (code=6 for BGR2GRAY) first.
• flow - NPARRAY. Here's the rough edge: flow is the function's out-parameter, but the generator surfaced it as a required input. You have to wire something even though it gets overwritten, and OpenCV expects a (H, W, 2) float32 array in that slot - which nothing in this pack conveniently produces. This is exactly the "ugly and complex to use" the README promises.
• pyr_scale (FLOAT) - pyramid scale, 0.5 is the standard.
• levels (INT) - pyramid levels; more levels = handles larger motions.
• winsize (INT) - averaging window; bigger = smoother but coarser, 15–21 is typical.
• iterations (INT) - 5 is a good default.
• poly_n / poly_sigma - polynomial window size (5 or 7) and sigma (1.1–1.5).
• flags (INT) - 0 for a fresh computation (4 = use a previous flow as the start).

Output: one nparray - the (H, W, 2) float flow field. It is not an image. Two-channel arrays don't convert cleanly through Nparrays2Image (which expects grayscale or 3-channel), so you'd need to visualize it yourself (e.g. polar-magnitude → hue), which this pack doesn't provide.

Install

ComfyUI Manager → opencv-comfyui, or git clone https://github.com/geroldmeisinger/opencv-comfyui into ComfyUI/custom_nodes, restart. Needs opencv-contrib-python.

The honest verdict

This is a node for people already doing CV math, not for beginners. The required-out-parameter flow slot and the non-image output mean real friction before you get a visible result. If your actual goal is motion-aware video processing, the dedicated video/AnimateDiff-adjacent ecosystems in ComfyUI have friendlier, purpose-built motion tooling. Farneback here is the raw scientific primitive: correct, powerful, and fully willing to make you work for it.