cv2.getOptimalNewCameraMatrix
The answer to black corners after undistort
- cameraMatrix
- distCoeffs
- imageSize
- newImgSize
- newCameraMatrix
- validPixROI
Undistort a wide-angle image and the barrel-distortion correction pulls the corners inward - you're left with black wedges where the sensor had pixels and the corrected frame doesn't. This node computes a new camera matrix that reframes the undistorted view sensibly, and hands you both that matrix and the rectangle of pixels that are actually valid in the result. It's the difference between "technically undistorted" and "usable".
The inputs that matter
cameraMatrix- the 3×3 intrinsicsKfrom your calibration.NPARRAY, and it must be the real one; a made-upKgives you a matrix that reframes imaginary pixels.CV Calibrate Camera (Chessboard)/(ChArUco)/(Circle Grid)produce it,CV Camera Matrixbuilds it by hand from fx/fy/cx/cy,CV Load Camera Params (JSON)reads it back from a file.distCoeffs- the distortion vector from the same calibration (k1, k2, p1, p2[, k3...]). It's a requiredNPARRAYlink in this pack's schema, and OpenCV treats an empty vector as zero distortion (an ideal pinhole camera), which is the fallback you want only if you genuinely calibrated nothing.imageSize- the original image size as aCV_TUPLE,(w, h), before any resizing. Get this wrong and the new matrix is scaled for a resolution you don't have.alpha- the whole point of the node.0keeps only valid pixels so there are no black borders (and zooms in on what's left);1keeps every source pixel so the framing matches the original (with black wedges); anything between interpolates.
Optional, and usually left alone: newImgSize ((w, h), defaults to imageSize) and centerPrincipalPoint (BOOLEAN, default False - the pack marks it as preset to the OpenCV default, which means the principal point gets chosen to best fit the valid region rather than forced to the image centre).
The outputs, and where they go
Two. newCameraMatrix is the reframed NPARRAY - the tooltip spells out the intended chain, and it's right: pass the original cameraMatrix, the distCoeffs, this new matrix and your size into initUndistortRectifyMap to produce maps, then remap through them. That's the standard undistort path, and doing it with maps (rather than one-shot undistort) is what lets you reuse the same maps across a video batch.
validPixROI is a BOUNDING_BOX - a cv2 Rect as core ComfyUI bbox data, nested one group per frame, exactly like every other box this pack emits. Its tooltip names the three consumers: CV Crop by BBoxes to trim the result to the valid region, Draw BBoxes to see where the region is, or CV Split Tuple if you just want x/y/w/h as numbers.
A note on the plumbing
imageSize and newImgSize are composite CV_TUPLE values, not loose integers: one value with two components, travelling as a whole. CV Tuple authors one (INT or float components, and it also emits the same numbers as an NPARRAY vector and as an (a, b) literal), and CV Split Tuple takes one apart. You can't half-connect a composite, which sounds restrictive until you've debugged the version of this where (w, h) arrived as (h, w).
Install
pip install "opencv-contrib-python-headless~=5.0.0.93"
Then ComfyUI Manager → search ComfyUI CV, or:
cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
and restart. Python ≥ 3.12 and a recent V3-API ComfyUI are required. No models, no downloads - this node is pure geometry on numbers you already have. (The pack's DNN examples do need external ONNX models, but nothing in the calibration lane does.)
Where people get burned
- Version drift. The pack is curated against OpenCV 5.0.0.93 and the README warns other versions may behave differently. If you've ended up with a stale 4.x wheel in the same venv, drop in
cv2.getVersionStringand look at it before blaming the node - and remember that all four OpenCV PyPI distributions share onesite-packages/cv2, so a non-contrib install over a contrib one silently strips the contrib submodules this pack's other nodes rely on.tools/repair_opencv_contrib.py --checkdiagnoses that. - Wrong
imageSize. The most common self-inflicted wound: square frames hide it, and then you feed the same workflow a 1920×1080 and the whole corrected view sits offset. - Expecting a magic
alpha. There's no setting that keeps every pixel and has no black borders - that's geometry, not a bug. Pick the trade, or crop withvalidPixROI.
One more piece of framing from the pack's own README, because it saves time: a lot of what this node feeds has a curated equivalent in the pack's 317 hand-written nodes (CV Fisheye Undistort, CV Omnidir Undistort, and friends), which handle the shape and flag traps for you. Use the raw wrapper when you're building something the curated nodes don't cover - and reach for the curated one when you just want an undistorted image.
Inputs (6)
| Name | Type | Default | Description |
|---|---|---|---|
| cameraMatrix | NPARRAY | Input camera intrinsic matrix. A data array (points / matrix), NOT an image - only an NPARRAY link is accepted here. | |
| distCoeffs | NPARRAY | Input vector of distortion coefficients $\distcoeffs$. If the vector is NULL/empty, the zero distortion coefficients are assumed. A data array (points / matrix), NOT an image - only an NPARRAY link is accepted here. | |
| imageSize | CV_TUPLE | 0,0 | Original image size. One value with 2 components (w, h) - it travels as a whole, so it cannot arrive half-connected. Wire it from 'CV Tuple' or type the components in place. |
| alpha | FLOAT | 0.0000-1e+38–1e+38 | Free scaling parameter between 0 (when all the pixels in the undistorted image are valid) and 1 (when all the source image pixels are retained in the undistorted image). See #stereoRectify for details. |
| newImgSizeopt | CV_TUPLE | 0,0 | Image size after rectification. By default, it is set to imageSize . One value with 2 components (w, h) - it travels as a whole, so it cannot arrive half-connected. Wire it from 'CV Tuple' or type the components in place. |
| centerPrincipalPointopt | BOOLEAN | false | Optional flag that indicates whether in the new camera intrinsic matrix the principal point should be at the image center or not. By default, the principal point is chosen to best fit a subset of the source image (determined by alpha) to the corrected image. Preset to the OpenCV default (False). |
Outputs (2)
| Name | Type | Description |
|---|---|---|
| newCameraMatrix | NPARRAY | — |
| validPixROI | BOUNDING_BOX | new_camera_matrix Output new camera intrinsic matrix. The function computes and returns the optimal new camera intrinsic matrix based on the free scaling parameter. By varying this parameter, you may retrieve only sensible pixels alpha=0 , keep all the original image pixels if there is valuable information in the corners alpha=1 , or get something in between. When alpha>0 , the undistorted result is likely to have some black pixels corresponding to "virtual" pixels outside of the captured distorted image. The original camera intrinsic matrix, distortion coefficients, the computed new camera intrinsic matrix, and newImageSize should be passed to #initUndistortRectifyMap to produce the maps for #remap . A cv2 Rect as core BOUNDING_BOX data ({x, y, width, height}, nested one group per frame) - feed 'Crop By Bounding Boxes', 'Draw BBoxes', or 'CV Split Tuple' for x/y/w/h. |