cv2.optflow.calcOpticalFlowSparseToDense

Most dense optical flow estimators work per pixel and drown in large motion. This one cheats, in a good way: it tracks a sparse grid of features between the two frames - Lucas-Kanade style, with descriptors to make the matching robust - and then fits a locally affine model through those matches to interpolate a dense (dx, dy) field everywhere. The pack's own tooltip sums it up as "tracks sparse features and interpolates them into a dense field - much faster than Farneback on large motion."

If your two frames differ by a nudge, Farneback is fine and simpler. If they differ by a lot - a fast pan, a jump cut you're trying to bridge, handheld shake that isn't small - this is the estimator with a chance.

The three inputs that matter

from_ and to are your two frames. A ComfyUI IMAGE links straight in (converted to 8-bit BGR; frame 0 if you feed a batch, since this isn't in the pack's per-frame-batch list). They must be the same size - flow is a per-pixel correspondence and there's nowhere for a size mismatch to hide.

Then the optional knobs, which are where the quality lives. These are advanced inputs (hidden until you show advanced inputs), and unlike the SimpleFlow wrapper they come pre-filled with the OpenCV defaults, because the author curated these:

• grid_step (8) - how far apart the tracked points are. Smaller = denser, better, slower. This is the one to move first.
• k (128) - nearest-neighbour matches considered per pixel when fitting the local affine model. Lower is noticeably faster at some quality cost.
• sigma (0.05) - how fast the interpolation weights fall off. Higher preserves fine detail; lower cleans up noise in the field.
• use_post_proc (on), with fgs_lambda (500) and fgs_sigma (1.5) - an edge-preserving fastGlobalSmootherFilter pass over the dense field. Leave it on unless you're debugging.

One thing to know about the output of that post-process: ximgproc.fastGlobalSmootherFilter is itself in the pack's always-offload list, so with post-processing on you're paying for a second heavy cv2 call. With it off, this node is markedly cheaper.

What comes out

One nparray, H x W x 2 float32 - (dx, dy) per pixel. Data, not a picture. Where it goes:

• CV Flow Map - adds the field onto a remap coordinate chain, then low-level cv2_remap warps the frame along its own motion. That's the compounding/retiming trick, and the pack's docs spell out the sign convention: remap samples backward, so a scale of -t pulls a frame a fraction t toward the next.
• CV Flow To Color - HSV wheel, for looking at it like a human.
• CV Draw Flow Grid - arrow grid over nothing in particular, useful for spotting degenerate fields.
• CV Draw Flow Vectors - arrows drawn on the frame itself.

For point-level analysis rather than pixel-level, the pack also has CV Detect Features + CV Match Features + CV Track Features (KLT), which keep the sparse side sparse.

Installing

ComfyUI Manager → search ComfyUI CV → Install → restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Python ≥ 3.12, a recent ComfyUI (V3 API), and the contrib OpenCV wheel - the auto-generated node list only contains functions your cv2 actually exposes, so with a core-only build this node simply isn't in the menu. Nothing to download model-wise.

Gotchas

It's slow, so it's out of process. optflow.calcOpticalFlowSparseToDense is not in the pack's offload list (unlike calcOpticalFlowSF and calcOpticalFlowFarneback), which means no ~1s spawn overhead - but also no cancel grace: an enormous grid_step=1 run on 4K frames is a call you get to wait out. Downscale first.

Silent single-frame behaviour. Feed a 10-frame IMAGE batch on each side and you get one field from frame 0 of each. The pack loops per frame only for a short list of cv2 functions, and this isn't on it. Pair frames explicitly with CV Index Batch if you're walking a clip.

The field composes with the remap chain, so check your units. Everything here is in pixels; if you've resized frames somewhere upstream, the field is in the resized pixel space. That mismatch shows up as a warp that's "almost right", which is the most annoying kind of wrong.

On the ecosystem side: this pack is a single-author, heavily AI-assisted 2026 fork with, as far as I can tell, no community threads at all. Its README is candid about that - the workflows are showcases, not production pipelines - so treat the curated nodes (which have found flags, sizes checks and error handling) as the safe layer and the raw cv2.* wrappers as the pair of tweezers: precise, sharp, and not always the tool you needed.