CV Optical Flow (DeepFlow / PCAFlow)

Two more dense-flow algorithms from cv2.optflow, sharing a node because their Python bindings expose no setters at all - there's nothing to tune, so there's nothing to configure beyond picking which one. That's unusual for this pack, where half the flow nodes have a screen of advanced parameters.

The author's own steer is to start with CV Optical Flow (DIS) and come here only for the two specific cases these cover.

Which one, and why

DeepFlow matches a descriptor pyramid before the variational solve. That extra matching stage is what lets it follow large displacements that purely pyramidal methods lose - a hard cut, a fast pan, a big jump between two frames that aren't adjacent. It's the slowest option in the node.

PCAFlow projects the field onto a learned low-dimensional basis. It's fast and it's immune to noise, because noise doesn't live in the basis. The cost is baked into the method: it can only produce motion its basis can express, so fine structure gets smoothed away. If your subject is a person walking, fine. If it's leaves in wind, don't expect the details.

So the decision is: motion too big → DeepFlow. Input too noisy → PCAFlow. Anything else → DIS.

Inputs and outputs

frame_a and frame_b are the two frames in time order. Both are polymorphic - an IMAGE or MASK drops in directly (frame 0 of a batch) or wire an NPARRAY - and both get converted to grayscale internally, which is why the flow is luminance-based even if you fed color. If the sizes differ, frame_b is resized to frame_a. algorithm is the DeepFlow / PCAFlow choice and the only other input.

Three outputs, and the set is standardized across every flow node in this pack: flow is HxWx2 float32 - the (dx, dy) displacement per pixel, the thing you'd feed into cv2_remap to actually warp something; magnitude is HxW float32 motion in pixels; angle is HxW float32 direction in radians, which is exactly what CV Flow To Color wants in its default mode. Sampling the same pixel in the flow field twice and comparing is how you get a motion measurement; CV Array Statistic in median mode is the usual way to pull a dominant field out of it.

Install

Manager → search ComfyUI CV; or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
cd comfyui_cv && pip install "opencv-contrib-python-headless~=5.0.0.93"

Restart. Python ≥ 3.12, ComfyUI with the V3 node API. Both algorithms live in cv2.optflow, which is contrib - on a non-contrib wheel this node either doesn't exist or errors on the algorithm. The pinned opencv-contrib-python-headless~=5.0.0.93 is the version the pack is curated against; other versions "may behave differently", in the README's words, which for a class-gated module like optflow is worth taking literally.

Common issues

• Instant error mentioning optflow - you're on a non-contrib OpenCV, or a plain opencv-python wheel clobbered the contrib one in site-packages/cv2. tools/repair_opencv_contrib.py --check from the pack reports it.
• Flow came back looking like a gray mush - PCAFlow smoothing, doing what it says. Switch to DeepFlow or DIS.
• Size mismatch complaints - check you fed the frames the right way round; frame_b gets resized to frame_a, so if you swapped them you get a flow that means the opposite direction.
• cv2 won't import on a Windows portable install - the mixed-wheel DLL failure a lot of ComfyUI users run into when several node packs each install their own OpenCV. It's an environment problem, and it takes out every cv2-based node in your install, not just this one.

For real video work, remember the flow field is the input to other things: warping, interpolation, stabilisation, or just measuring how much the camera moved between two frames.