cv2.fisheye.stereoRectify

Stereo matching has one hard requirement: a point in the left image must lie on the same row in the right image. Real cameras don't cooperate, and fisheye cameras cooperate even less, because the distortion is radial and huge. cv2.fisheye.stereoRectify computes the pair of rotations and new projection matrices that warp both views into a shared, row-aligned frame under the equidistant fisheye model. It's the stage before disparity: rectified pair in, a disparity node (the pack's CV Stereo Disparity BM or CV Stereo Disparity / SGBM nodes) out, depth from Q.

It's a raw wrapper in ComfyUI CV (bmad4ever) - the pack that exposes OpenCV 5.0 as ComfyUI nodes. Which means: OpenCV's arguments, OpenCV's defaults, and no hand-holding.

What it gives you

Five NPARRAY outputs, in cv2's order:

• nparray_0 / nparray_1 - R1, R2: the rectification rotations for camera 1 and camera 2.
• nparray_2 / nparray_3 - P1, P2: the new (rectified) projection matrices.
• nparray_4 - Q, the 4×4 disparity-to-depth matrix. Feed it to cv2.reprojectImageTo3D together with a disparity map and you get a 3D point cloud; that's the whole point of the exercise.

Note what's not here: no images. This node only computes geometry. The actual rectifying warp is the next step - cv2.fisheye.initUndistortRectifyMap(K, D, R, P, size, m1type) to build a map pair, then cv2.remap to apply it, once per camera. Both are separate nodes in the pack.

The inputs

• K1 / D1 / K2 / D2 - each camera's intrinsics and four fisheye coefficients. These come from a stereo calibration of the same image size; a mismatch between the calibration size and the size you pass here is the classic silent failure.
• imageSize - a required CV_TUPLE with two components (w, h). It travels as one value rather than two loose numbers, so it can't be half-connected; the tooltip says to wire it from CV Tuple or type the components in place.
• R, tvec - the relative rotation (3×3) and translation between the two cameras, from the stereo calibration step.
• flags - a plain INT, default 0, and there's no friendly dropdown here (the pack does build a CALIB_ZERO_DISPARITY combo for the pinhole cv2.stereoRectify, but not for this one, because OpenCV 5 moved the fisheye CALIB_* constants into the shared cv2.CALIB_* namespace and their values changed from the 4.x fisheye ones - the pack's own flag table notes CALIB_RECOMPUTE_EXTRINSIC was 2 in 4.x and is 1<<23 here). Do not paste a number out of a 4.x tutorial. 0 means no flags, which is not the same as zero-disparity.
• newImageSize, balance, fov_scale - optional, advanced, and only meaningful in the zero-disparity path; if they look inert, check the flag bit first.

Install

ComfyUI Manager → ComfyUI CV, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install -r comfyui_cv/requirements.txt

Python ≥3.12 and recent ComfyUI (V3 API). Restart. The dependency is opencv-contrib-python-headless~=5.0.0.93; on a different OpenCV build these fisheye nodes may simply not exist, because the pack generates its wrappers from whatever your installed build exposes.

Common issues

"The category is missing" / cv2.fisheye is an empty module. Installing plain opencv-python over the contrib wheel empties the contrib submodules without uninstalling anything, because all four distributions share one site-packages/cv2. Diagnose and repair:

python ComfyUI/custom_nodes/comfyui_cv/tools/repair_opencv_contrib.py --check
python ComfyUI/custom_nodes/comfyui_cv/tools/repair_opencv_contrib.py --apply

Rows still don't line up after rectifying. Usually a calibration problem, not a rectify problem: K/D from a pinhole calibrator (cv2.calibrateCamera) describes a different model than K/D from the fisheye one, and mixing them produces plausible geometry that doesn't quite land. The pack's fisheye workflow measures this explicitly - on its own synthetic views, the pinhole calibrator also fits them to 0.13 px RMS, because over the middle of the frame five Brown–Conrady coefficients can imitate a fisheye. The models disagree where there's no data.

Note this is a single-frame node in practice. It isn't in the pack's per-frame batch list, so a multi-frame IMAGE link goes through as frame 0. If you're rectifying a sequence, drive the loop yourself or use the pack's batch bridge nodes; and remember the stereo disparity nodes in the same pack have their own playground workflow (58_stereo_bm_vs_sgbm.json) if you want a reference pair to compare BM against SGBM on.