CV Omnidir Stereo Reconstruct

Stereo depth on normal cameras is a solved, boring problem - rectify, match, triangulate. Do the same thing with a pair of catadioptric rigs (the "mirror ball on a lens" cameras that see nearly the whole sphere) and the rectification stops being a homography, which is why you can't just wire this together out of pinhole nodes. CV Omnidir Stereo Reconstruct wraps cv2.omnidir.stereoReconstruct, which does the rectify + SGBM match + back-projection in one call.

Worth being honest about where this sits in 2026. If you just want a depth map or a mesh from pictures, a learned model will beat this and you should use that instead - Depth Anything 3 and friends hand you consistent geometry from any number of views with no calibration at all. This node is for the case where you own the rig, you ran the calibration, and you want real metric geometry from two omnidirectional images rather than a plausible-looking monocular guess.

How it works

You feed both cameras' intrinsics K_left / K_right, both mirror parameters xi_left / xi_right, both 4-coefficient distortion vectors dist_left / dist_right, and the R / T between the two cameras. Add two knobs for the matcher: num_disparities (the SGBM search range, a multiple of 16) and block_size (the matching window, odd).

projection picks the frame the matcher runs in. RECTIFY_PERSPECTIVE keeps the middle of the field at higher resolution; LONGLATI unrolls the whole sphere and matches more of it at lower resolution per pixel. cv2 only accepts those two here, so don't go looking for cylindrical.

Two practical things the node does for you. The disparity it returns is in the rectified frame, not the frame you fed in - compare it against left_rectified, not your original image. And where cv2 writes -1 for "nothing matched", this node writes 0 and hands you a separate valid mask, so you can't accidentally average a sentinel value into a depth estimate.

Inputs and outputs that matter

The one you'll actually tune is num_disparities: it sets how near the pair can see. Too small and the foreground clips at a flat wall of failed matches. block_size trades detail for smoothness - 7 is a sane start, bigger gets you smoother and blinder.

Six things come out. disparity (float32, rectified frame, 0 where nothing matched) and valid (uint8, 255 where the disparity is real) are the raw results. left_rectified / right_rectified are the frames everything is aligned to. point_cloud is the Nx3 float32 cloud in the left camera's frame, in the units of T - feed it to CV Filter Point Cloud or CV Point Cloud Normals. And found is the boolean to branch on: a cv2 error returns empty outputs with found=false instead of blowing up your run.

Install

From ComfyUI Manager, search for the pack title ComfyUI CV and install, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
cd comfyui_cv && pip install "opencv-contrib-python-headless~=5.0.0.93"

Then restart ComfyUI. The pack needs Python ≥ 3.12 and a recent ComfyUI built on the V3 node API. cv2.omnidir lives in the contrib build - if you've ever installed plain opencv-python over a contrib wheel, the contrib submodules go empty and this node's whole module disappears. tools/repair_opencv_contrib.py --check diagnoses that; --apply fixes it.

Common issues

• Everything comes back empty, found=false. Most often a size mismatch between the pair, or a dist_left / dist_right with five values in it. Omnidir coefficients are four (k1, k2, p1, p2); a 5-element vector is the pinhole set, and the node says so in the error.
• The cloud is in nonsense units. It's in the units of T, always. Rescale T and you rescale the world.
• The disparity looks aligned to nothing. You're comparing it to the input image. Compare it to left_rectified.
• cv2 refuses to import at all (DLL load failed) - the classic Windows-portable, mixed-wheel mess reported by other ComfyUI users when two node packs disagree about which OpenCV wheel should own site-packages/cv2. Fix the environment, not the graph.

One last caveat from the pack's own README, and it's real: this whole pack was written with heavy LLM assistance and the author says outright it isn't production-ready and updates aren't planned. Verify anything you'd bet a pipeline on.