Nodes/opencv-comfyui/OpenCV stereoRectify_0
ComfyUI Node

OpenCV stereoRectify_0

Calibrated stereo rectification — for the 0.1% doing real 3D

By geroldmeisinger·Created about a year ago·Updated about a year ago· 35
OpenCV stereoRectify_0
  • cameraMatrix1
  • distCoeffs1
  • cameraMatrix2
  • distCoeffs2
  • R
  • T
  • R1
  • R2
  • P1
  • P2
  • Q
  • nparray_0
  • nparray_1
  • nparray_2
  • nparray_3
  • nparray_4
  • literal_5
  • literal_6
imageSize
flags
alpha
newImageSize

Let's be honest about who this node is for: you, if you're doing actual stereo depth - two calibrated cameras, disparity maps, real 3D reconstruction - and almost nobody else. stereoRectify is the workhorse of calibrated stereo vision: you give it the intrinsics and distortion of both cameras plus the rotation/translation between them, and it hands back the rectification transforms that make matching a solved problem. It's not a "make it prettier" node. It's camera math, and it assumes you already did a calibration pass.

What rectification buys you: after applying these transforms, corresponding points in the two images sit on the same horizontal scanline, so stereo matching collapses from a 2D search to a 1D one. That's the step before running a block-matching stereo algorithm to get disparity, which you then turn into depth with the Q matrix. (Heads-up: this pack only wraps standalone cv2 functions, so the StereoBM/StereoSGBM classes aren't in it - you'd run the matching step in your own cv2 code.) In ComfyUI terms, you're building a stereo-depth pipeline from raw camera data instead of using a depth-estimation model. That's a wildly different thing to be doing in a diffusion UI, but the pack deliberately ships most of OpenCV, so here it is.

Inputs that matter:

  • cameraMatrix1 / cameraMatrix2 - the 3×3 intrinsics of each camera, as NPARRAY.
  • distCoeffs1 / distCoeffs2 - distortion coefficients, 4–5 element NPARRAY (or an empty array if undistorted).
  • R, T - rotation and translation between the cameras.
  • imageSize - STRING literal, e.g. (640, 480).
  • flags - INT; 0 means the standard CALIB_ZERO_DISPARITY-free default.
  • alpha - FLOAT; 0 crops to valid pixels, 1 keeps everything, -1 keeps it all.
  • newImageSize - STRING literal, often same as imageSize.

Outputs are five NPARRAYs - R1, R2, P1, P2, Q - plus two STRING literals that are the stringified valid-pixel ROI rectangles. R1/R2 and P1/P2 feed this pack's initUndistortRectifyMap_0 to build remap maps (or cv2.remap directly), and Q is what converts disparity to 3D. stereoRectify_1 is the identical twin, same generated wrapper, pick either.

The traps. First, every matrix input is NPARRAY - this pack's numpy/BGR convention, not ComfyUI tensors. If you're building matrices by hand, construct them in numpy and feed them as arrays; there's no friendly widget. Second, imageSize and newImageSize are STRING literals - type (640, 480), not 640, 480, or you'll get invalid syntax (<unknown>, line 0). Third, the pack only handles batch_size 1 images, which is fine here since you're not feeding images at all.

Install. ComfyUI Manager → search "opencv-comfyui", or cd ComfyUI/custom_nodes && git clone https://github.com/geroldmeisinger/opencv-comfyui, restart. Needs opencv-contrib-python, numpy, torch; the contrib wheel matters because several camera functions live there. If you see Cannot import name 'guidedFilter' from 'cv2.ximgproc', fix the duplicate OpenCV installs first. And set expectations: this is auto-generated, unpolished camera math in a node graph - the author's own tagline is "Expect dragons!", and this node is deep in dragon country.

Categoryimage/OpenCV

Inputs (15)

NameTypeDefaultDescription
cameraMatrix1NPARRAY
distCoeffs1NPARRAY
cameraMatrix2NPARRAY
distCoeffs2NPARRAY
imageSizeSTRING
RNPARRAY
TNPARRAY
flagsINT
alphaFLOAT
newImageSizeSTRING
R1optNPARRAY
R2optNPARRAY
P1optNPARRAY
P2optNPARRAY
QoptNPARRAY

Outputs (7)

NameTypeDescription
nparray_0NPARRAY
nparray_1NPARRAY
nparray_2NPARRAY
nparray_3NPARRAY
nparray_4NPARRAY
literal_5STRING
literal_6STRING