cv2.rapid.findCorrespondencies

What this node actually is

This is one step of RAPID, OpenCV's implementation of Harris & Stennett's 1990 appearance-based tracker. The idea, in one breath: you already have a 3D model, so you can project it and you know roughly where its silhouette should be. Take the projected control points, shoot a search line perpendicular to the contour through each one, and look for the strongest image gradient along that line. Those hits are your 2D→3D correspondences, and they let you refine the pose with PnP.

cv2.rapid.findCorrespondencies is that middle step: it walks the search lines and reports what it found. It's a raw wrapper, auto-generated from the type stubs, sitting in the pack's image/CV/low-level/rapid category.

First thing to know: the weird spelling is OpenCV's, not the pack's. The function really is findCorrespondencies (the "dences" one), so if you googled the correctly-spelled name and found nothing, that's why. The second thing: this is a tracker component, not a detector. Nothing here finds an object in a scene - it refines a guess that's already close.

The one input, and why it's one input

The node takes exactly one thing: bundle. That surprises people, because you'd expect an image. The bundle is the RAPID line-bundle struct built by cv2.rapid.extractLineBundle (len, ctl2d, img → bundle, locations) - it already holds the image, the control points and the search lines. So the wiring goes: cv2.rapid.extractControlPoints → control points, extractLineBundle → bundle, then this node.

The socket is typed to accept an image or a mask thanks to the pack's generic image-input plumbing, but don't: a bundle is an opaque array, and handing it a picture will just blow up in cv2. Wire it from extractLineBundle or nothing.

Outputs:

• cols - the correspondences: which control point matched where along its line.
• scores - how strong each match was. Useful as a filter: a frame where the scores are all mushy is a frame where the tracker is guessing.

From here the chain is cv2.rapid.convertCorrespondencies (cols plus the locations from extractLineBundle, optional mask) → pts2d + mask, then a cv2.solvePnPRansac to get the refined pose. That's the full RAPID loop, and it's why the pack ships a subgraph - CV Rapid Pose Refine - that folds this node plus the PnP into one reusable block. If you just want tracked poses for a clip, CV Rapid Track (Sequence) does the whole thing over an image batch and gives you per-frame rvec/tvec, so you never touch this node directly.

Reach for the raw wrapper when you're debugging a track - dumping scores over time tells you whether the silhouette or the background is winning - or when you want to substitute your own PnP.

Install

ComfyUI Manager → search "ComfyUI CV" (author: bmad4ever). Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Restart ComfyUI afterwards. The pack wants Python ≥ 3.12 and a ComfyUI new enough to have the V3 node API (comfy_api.latest) - on an older ComfyUI the nodes simply don't register.

Where people get burned

RAPID lives in a contrib submodule, so a non-contrib OpenCV makes it vanish. The pack's own README flags this: installing opencv-python or opencv-python-headless over a contrib build silently empties the contrib submodules, and cv2.rapid goes with them. The node disappears from the menu and a workflow referencing it opens red. Check before you debug anything else:

python -c "import cv2; print(cv2.__version__); print(cv2.rapid.rapid)"

If that raises AttributeError, repair the wheel with the pack's own script:

python ComfyUI/custom_nodes/comfyui_cv/tools/repair_opencv_contrib.py --check
python ComfyUI/custom_nodes/comfyui_cv/tools/repair_opencv_contrib.py --apply

And treat the output as evidence, not truth. findCorrespondencies takes the strongest gradient anywhere on the line, which is how texture beats silhouette. The pack's own notes on the sequence node measured a textured vehicle over clutter pinning at ~46 px of error when the search line got long. If your track drifts, the fix is upstream (shorter search lines, a downscaled frame, a denser mesh), not in this node.