CV Optical Flow (RLOF, Dense)

Robust Local Optical Flow tracks a grid of points with an illumination-aware local solver, then interpolates those sparse results into a dense field using the same edge-aware interpolators that the disparity nodes use. Two consequences, and they're the whole reason to pick it.

First, it's the best of the classic methods when your two frames differ in exposure. The brightness/contrast change is part of the per-point solve (use_illumination_model, on by default), so an auto-exposure flicker or a flash in one frame of the pair doesn't wreck the motion estimate the way it does for brightness-constancy methods.

Second - and this is the surprise - this node needs colour. The cross-based support region it grows around each point segments on colour similarity. Feed it grayscale and the pack expands it to BGR, which means the method loses its main advantage and you've paid for nothing. If your source is grayscale, use DIS instead.

The knob that matters and the one that bites

grid_step (default 6) is the spacing of the point grid that's actually tracked; everything between points is interpolated. It's the dominant speed/detail control. Lower for detail, raise for speed. If you only touch one thing, touch this.

interpolation decides how the grid becomes dense:

• EPIC (default) - fits a local affine model along image edges.
• RIC - one plane per SLIC superpixel, sharpest at motion boundaries and the slowest. Worth knowing: it randomizes its model fitting, so its output varies slightly between identical runs. If you're chasing a reproducible number, that's a trap.
• GEO - plain geodesic weighting, fastest.

Then forward_backward_threshold (default 1 px) drops grid points that fail a re-tracking consistency check - set it to 0 and you keep the occluded ones, which is rarely what you want.

use_variational_refinement runs a polish over the interpolated field; off by default, and it costs. Of the advanced block, use_post_proc (on by default) is worth leaving on - without the fast global smoother the per-region models stay visible as visible blocky steps. Everything else there (epic_k, epic_sigma, epic_lambda, the ric_* superpixel options, small_win_size / large_win_size, max_level, max_iterations, min_eigen_value, support_region, solver_type, cross_segmentation_threshold, use_global_motion_prior, global_motion_ransac_threshold, use_m_estimator, fgs_lambda, fgs_sigma) is a real parameter you only touch if you've measured a reason.

use_m_estimator deserves a note: the Lorentzian robust norm is off here, matching cv2's own default (the parameter object ships with the norm sigmas at FLT_MAX). Turning it on costs time and helps when the two frames contain genuine outlier motion.

Inputs and outputs

frame_a and frame_b, both polymorphic (IMAGE/MASK direct or NPARRAY), kept in 8-bit colour, frame_b same size as frame_a. Outputs are the pack's standard flow trio: flow (HxWx2 float32), magnitude (HxW float32, pixels), angle (HxW float32, radians - the default input mode for CV Flow To Color).

Install

Manager → ComfyUI CV, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
cd comfyui_cv && pip install "opencv-contrib-python-headless~=5.0.0.93"

Restart. Python ≥ 3.12, V3 node API. RLOF lives in cv2.optflow - contrib only. If a non-contrib wheel has ever owned site-packages/cv2, this node will fail on the module rather than give you a bad result. tools/repair_opencv_contrib.py --check in the pack tells you which wheel you're actually running.

Common issues

• The flow is fine but the boundaries are soft - RIC is likely the fix, at a real time cost, and accept that it's non-deterministic. Also make sure your input is actually colour.
• Weird blocky steps in the field - use_post_proc got turned off. Put it back.
• Grayscale input, disappointing result - expected, as above. This method's support-region segmentation is colour-driven.
• Points vanishing in the foreground - that's forward_backward_threshold doing its job on occlusion. Set it to 0 if you'd rather keep them.
• cv2 won't import at all - the classic multi-pack OpenCV wheel collision on Windows portable installs. Environment problem, takes out every cv2 node in the graph, fix the environment.

The README is honest that this is a personal project with heavy LLM assistance and no production support - the algorithm is OpenCV's, but check anything you're going to depend on.