cv2.calcOpticalFlowFarneback
Dense motion, and the seven numbers that default to 0
- prev
- next
- flow
- nparray
Farneback's algorithm answers "where did every single pixel go between these two frames?" - one motion vector per pixel, dense, no keypoints, no model. That's what this node computes. It's the classic pre-deep-learning dense flow, and for a lot of jobs (building a motion mask, judging how much a frame pair actually moved, driving a warp) it's still the cheapest correct answer.
The node is a raw wrapper from comfyui_cv (~470 cv2.* nodes for ComfyUI). Two things to know before wiring anything: the parameters are required and therefore not filled with OpenCV's sensible defaults, and the curated version of this same function ships in the pack too.
Read this part first: the defaults are zeros
The pack fills in optional parameters for you and leaves them blank when it can't. These aren't optional in the type stubs, so all seven knobs land at 0 - and pyr_scale=0, levels=0, winsize=0 is not a working configuration. You type OpenCV's own defaults in yourself:
| Input | Set it to |
| --- | --- |
| pyr_scale | 0.5 (classical pyramid, each level half size) |
| levels | 3 |
| winsize | 15 (larger = more robust to noise, blurrier motion) |
| iterations | 3 |
| poly_n | 5 (7 for smoother fields) |
| poly_sigma | 1.1 (use 1.5 if poly_n=7) |
The tooltips carry OpenCV's own guidance - poly_n and poly_sigma are a matched pair, larger means "smoother, more robust, blurrier". Leave flags on none (0); it's a proper dropdown with toggles for OPTFLOW_USE_INITIAL_FLOW and OPTFLOW_FARNEBACK_GAUSSIAN, and the Gaussian variant is slower but more accurate.
Inputs, and the buffer you must supply
prev,next(required) - 8-bit single-channel images of identical size. Single-channel is not a suggestion: an IMAGE link arrives as 3-channel uint8 BGR and cv2 rejects it. Feed a MASK, or a grayscale array you built with the pack'scv2.cvtColorwrapper.flow(required, NPARRAY) - the output buffer, and yes, it's an input. OpenCV expects caller-allocated space here, so the pack declares it as a required NPARRAY socket and you hand it an array of the right shape (the same[H, W, 2]float32 array the function will fill). This is whatcv2.broadcastand friends are for.
Output is a single nparray: a CV_32FC2 field - every pixel carrying (dx, dy). It is not displayable. To see it, use the pack's visualisation nodes: CV Flow To Color (angle → HSV colour wheel), CV Draw Flow Grid (arrows), or CV Flow Map. To do maths on it, split the two channels with cv2.extractChannel and then take magnitudes and angles with cv2.cartToPolar.
One batch caveat that bites video workflows: an IMAGE or MASK link gives you frame 0 of the batch. Dense flow is a per-pair computation, so a 100-frame clip doesn't magically produce a flow "video" from one call - you drive it per pair, or use the pack's curated flow nodes.
The honest recommendation
If you just want motion visible in an image, skip this node. The pack ships CV Optical Flow (Farneback) - "dense flow, free-function", with the plumbing and the buffer handled - and CV Flow To Color to render it. Same author, same algorithm, no zeros to fix.
Reach for the raw wrapper when you want the field as data: to threshold motion into a mask, to feed cartToPolar for a direction map, to combine with a warp, or to compare a parameter sweep. That's the difference between showing motion and measuring it.
Install
cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
Restart, or ComfyUI Manager → search "comfyui_cv". Python ≥ 3.12, recent ComfyUI on the V3 node API, and:
pip install "opencv-contrib-python-headless~=5.0.0.93"
No models involved - Farneback is plain OpenCV.
Common issues
cv2 assertion about the input type. You fed colour. Both frames must be single-channel 8-bit.
A black or uniform flow field. The parameter zeros. levels=0 and winsize=0 won't track anything.
cv2 complains that flow has the wrong size or type. The buffer has to match prev's dimensions exactly, with 2 float32 channels.
Motion comes out backwards in y. OpenCV's y axis grows downward, so a positive dy means down the frame. That's not a bug, it's image coordinates, and it's the same convention cartToPolar will hand you angles in.
Inputs (10)
| Name | Type | Default | Description |
|---|---|---|---|
| prev | NPARRAY,IMAGE,MASK | first 8-bit single-channel input image. Accepts a ComfyUI IMAGE/MASK directly (frame 0 of a batch) or an NPARRAY. Arithmetic ops (add, multiply, etc.) process the full IMAGE batch when both inputs have the same batch size. | |
| next | NPARRAY,IMAGE,MASK | second input image of the same size and the same type as prev. Accepts a ComfyUI IMAGE/MASK directly (frame 0 of a batch) or an NPARRAY. Arithmetic ops (add, multiply, etc.) process the full IMAGE batch when both inputs have the same batch size. | |
| flow | NPARRAY,IMAGE,MASK | computed flow image that has the same size as prev and type CV_32FC2. Accepts a ComfyUI IMAGE/MASK directly (frame 0 of a batch) or an NPARRAY. Arithmetic ops (add, multiply, etc.) process the full IMAGE batch when both inputs have the same batch size. | |
| pyr_scale | FLOAT | 0.0000-1e+38–1e+38 | parameter, specifying the image scale (<1) to build pyramids for each image; pyr_scale=0.5 means a classical pyramid, where each next layer is twice smaller than the previous one. |
| levels | INT | 0-2147483648–2147483647 | number of pyramid layers including the initial image; levels=1 means that no extra layers are created and only the original images are used. |
| winsize | INT | 0-2147483648–2147483647 | averaging window size; larger values increase the algorithm robustness to image noise and give more chances for fast motion detection, but yield more blurred motion field. |
| iterations | INT | 0-2147483648–2147483647 | number of iterations the algorithm does at each pyramid level. |
| poly_n | INT | 0-2147483648–2147483647 | size of the pixel neighborhood used to find polynomial expansion in each pixel; larger values mean that the image will be approximated with smoother surfaces, yielding more robust algorithm and more blurred motion field, typically poly_n =5 or 7. |
| poly_sigma | FLOAT | 0.0000-1e+38–1e+38 | standard deviation of the Gaussian that is used to smooth derivatives used as a basis for the polynomial expansion; for poly_n=5, you can set poly_sigma=1.1, for poly_n=7, a good value would be poly_sigma=1.5. |
| flags | STRING | none (0) | operation flags that can be a combination of the following: - **OPTFLOW_USE_INITIAL_FLOW** uses the input flow as an initial flow approximation. - **OPTFLOW_FARNEBACK_GAUSSIAN** uses the Gaussian $\texttt{winsize}\times\texttt{winsize}$ filter instead of a box filter of the same size for optical flow estimation; usually, this option gives z more accurate flow than with a box filter, at the cost of lower speed; normally, winsize for a Gaussian window should be set to a larger value to achieve the same level of robustness. The function finds an optical flow for each prev pixel using the algorithm so that $$\texttt{prev} (y,x) \sim \texttt{next} ( y + \texttt{flow} (y,x)[1], x + \texttt{flow} (y,x)[0])$$ cv2.calcOpticalFlowFarneback flags: one of none (0) plus any of OPTFLOW_USE_INITIAL_FLOW, OPTFLOW_FARNEBACK_GAUSSIAN, pipe-joined (e.g. "none (0) | OPTFLOW_USE_INITIAL_FLOW"). In the UI this renders as a dropdown with one toggle per flag. |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| nparray | NPARRAY | — |