cv2.stereoRectify

Stereo depth only works on pairs whose epipolar lines are horizontal and aligned. Your camera pair is not aligned like that - no real rig is. cv2.stereoRectify computes the two rotations and new projection matrices that force every matching feature onto the same row, which is the precondition for block matching, for reprojectImageTo3D, and for everything downstream of a depth map.

It is a plumbing node, and it's the kind that punishes guessing. Everything it needs comes from a calibration, and everything it produces is a matrix or a rectangle, not a picture.

This is one of roughly 470 auto-generated raw cv2.* wrappers in ComfyUI CV (bmad4ever/comfyui_cv). The pack's honest framing applies here more than anywhere: these wrappers are LLM-written, uncurated, and the author recommends not using them in production without reviewing the code yourself.

Inputs: all matrices, and NPARRAY only

Six required inputs, and every one of them is a data array, not an image - the sockets accept NPARRAY links only, and the tooltips say so outright:

• cameraMatrix1, cameraMatrix2 - the two 3x3 intrinsics. CV Stereo Calibrate (Chessboard) or CV Camera Matrix for a hand-typed guess.
• distCoeffs1, distCoeffs2 - the lens distortion vectors.
• R, T - rotation and translation between the two cameras, in each other's frame.
• imageSize - a (width, height) CV_TUPLE value. Wire it from a CV Tuple or off an image's measured size; (0, 0) is the default and doing anything real requires the actual size of the pair you calibrated on.

Three optional knobs, and the pack's tooltips are clearer than OpenCV's own docs on two of them. flags defaults to CALIB_ZERO_DISPARITY, which "makes the principal points coincide in the rectified views"; switch it to none if you'd rather "minimize unusable image area". alpha is the crop-versus-black-borders dial: 0 keeps only valid pixels (zoomed, no black borders), 1 keeps every source pixel (black borders), in-between interpolates, and the default -1 lets OpenCV pick. newImageSize at (0, 0) means "same as imageSize"; a larger value preserves more detail when the distortion is heavy.

Outputs

Five matrices: R1, R2 (the rectification rotations), P1, P2 (the new projection matrices) and Q, the 4x4 disparity-to-depth matrix. Plus two typed rectangles, roi1 and roi2, which come out as core BOUNDING_BOX data - feed them into Crop By Bounding Boxes or Draw BBoxes to see or use the valid region that alpha produced. That pair is the nicest part of the node: the rectangle you need to crop away the black wedge is handed to you in a format the rest of ComfyUI already understands.

From there, R1/P1 go into cv2.initUndistortRectifyMap and the two maps into cv2.remap, which is the actual rectification of the pixels. Q goes into cv2.reprojectImageTo3D along with a disparity map to give you a point cloud.

Reading a working example

The pack ships graphs that use this node, and they're worth opening before you wire your own: 52_circle_grid_calibration.json, 64_ar_depth_occlusion.json, and the exercise_stereo_multiview / exercise_stereo_pointcloud pair. In 64 the six matrices arrive from CV Stereo Calibration From CSV and imageSize from CV Array Size - that's the shape of a real pipeline. The author also warns that the example stereo settings are tuned to the StereoGeo-CARLA dataset, so port the structure, not the numbers.

Installing the pack

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
cd comfyui_cv && pip install "opencv-contrib-python-headless~=5.0.0.93"

Manager → search ComfyUI CV → install → restart is the usual route. Requires Python ≥ 3.12 and a ComfyUI with the V3 node API. The OpenCV wheel must be contrib: all four distributions share one site-packages/cv2, so installing plain opencv-python over the contrib build empties the contrib submodules and the contrib nodes vanish (tools/repair_opencv_contrib.py --check, then --apply). Pinned and tested against 5.0.0.93.

Where it bites

Passing (0, 0) for imageSize and getting a garbage Q - the size must match the pair you calibrated with, in pixels. Feeding a data-array socket from an IMAGE: these inputs are NPARRAY-only by design, and an image is not a matrix. And expecting the node to move pixels: it doesn't. It computes the geometry; remap does the work.