Nodes/ComfyUI CV/cv2.getOptimalNewCameraMatrix
ComfyUI Node

cv2.getOptimalNewCameraMatrix

The answer to black corners after undistort

By bmad4ever·Created 4 months ago·Updated 15 days ago· 1
cv2.getOptimalNewCameraMatrix
  • cameraMatrix
  • distCoeffs
  • imageSize
  • newImgSize
  • newCameraMatrix
  • validPixROI
◄alpha0.0000►
◄centerPrincipalPointfalse►

Undistort a wide-angle image and the barrel-distortion correction pulls the corners inward - you're left with black wedges where the sensor had pixels and the corrected frame doesn't. This node computes a new camera matrix that reframes the undistorted view sensibly, and hands you both that matrix and the rectangle of pixels that are actually valid in the result. It's the difference between "technically undistorted" and "usable".

The inputs that matter

  • cameraMatrix - the 3×3 intrinsics K from your calibration. NPARRAY, and it must be the real one; a made-up K gives you a matrix that reframes imaginary pixels. CV Calibrate Camera (Chessboard) / (ChArUco) / (Circle Grid) produce it, CV Camera Matrix builds it by hand from fx/fy/cx/cy, CV Load Camera Params (JSON) reads it back from a file.
  • distCoeffs - the distortion vector from the same calibration (k1, k2, p1, p2[, k3...]). It's a required NPARRAY link in this pack's schema, and OpenCV treats an empty vector as zero distortion (an ideal pinhole camera), which is the fallback you want only if you genuinely calibrated nothing.
  • imageSize - the original image size as a CV_TUPLE, (w, h), before any resizing. Get this wrong and the new matrix is scaled for a resolution you don't have.
  • alpha - the whole point of the node. 0 keeps only valid pixels so there are no black borders (and zooms in on what's left); 1 keeps every source pixel so the framing matches the original (with black wedges); anything between interpolates.

Optional, and usually left alone: newImgSize ((w, h), defaults to imageSize) and centerPrincipalPoint (BOOLEAN, default False - the pack marks it as preset to the OpenCV default, which means the principal point gets chosen to best fit the valid region rather than forced to the image centre).

The outputs, and where they go

Two. newCameraMatrix is the reframed NPARRAY - the tooltip spells out the intended chain, and it's right: pass the original cameraMatrix, the distCoeffs, this new matrix and your size into initUndistortRectifyMap to produce maps, then remap through them. That's the standard undistort path, and doing it with maps (rather than one-shot undistort) is what lets you reuse the same maps across a video batch.

validPixROI is a BOUNDING_BOX - a cv2 Rect as core ComfyUI bbox data, nested one group per frame, exactly like every other box this pack emits. Its tooltip names the three consumers: CV Crop by BBoxes to trim the result to the valid region, Draw BBoxes to see where the region is, or CV Split Tuple if you just want x/y/w/h as numbers.

A note on the plumbing

imageSize and newImgSize are composite CV_TUPLE values, not loose integers: one value with two components, travelling as a whole. CV Tuple authors one (INT or float components, and it also emits the same numbers as an NPARRAY vector and as an (a, b) literal), and CV Split Tuple takes one apart. You can't half-connect a composite, which sounds restrictive until you've debugged the version of this where (w, h) arrived as (h, w).

Install

pip install "opencv-contrib-python-headless~=5.0.0.93"

Then ComfyUI Manager → search ComfyUI CV, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv

and restart. Python ≥ 3.12 and a recent V3-API ComfyUI are required. No models, no downloads - this node is pure geometry on numbers you already have. (The pack's DNN examples do need external ONNX models, but nothing in the calibration lane does.)

Where people get burned

  • Version drift. The pack is curated against OpenCV 5.0.0.93 and the README warns other versions may behave differently. If you've ended up with a stale 4.x wheel in the same venv, drop in cv2.getVersionString and look at it before blaming the node - and remember that all four OpenCV PyPI distributions share one site-packages/cv2, so a non-contrib install over a contrib one silently strips the contrib submodules this pack's other nodes rely on. tools/repair_opencv_contrib.py --check diagnoses that.
  • Wrong imageSize. The most common self-inflicted wound: square frames hide it, and then you feed the same workflow a 1920×1080 and the whole corrected view sits offset.
  • Expecting a magic alpha. There's no setting that keeps every pixel and has no black borders - that's geometry, not a bug. Pick the trade, or crop with validPixROI.

One more piece of framing from the pack's own README, because it saves time: a lot of what this node feeds has a curated equivalent in the pack's 317 hand-written nodes (CV Fisheye Undistort, CV Omnidir Undistort, and friends), which handle the shape and flag traps for you. Use the raw wrapper when you're building something the curated nodes don't cover - and reach for the curated one when you just want an undistorted image.

Categoryimage/CV/low-level/cv2 G

Inputs (6)

NameTypeDefaultDescription
cameraMatrixNPARRAYInput camera intrinsic matrix. A data array (points / matrix), NOT an image - only an NPARRAY link is accepted here.
distCoeffsNPARRAYInput vector of distortion coefficients $\distcoeffs$. If the vector is NULL/empty, the zero distortion coefficients are assumed. A data array (points / matrix), NOT an image - only an NPARRAY link is accepted here.
imageSizeCV_TUPLE0,0Original image size. One value with 2 components (w, h) - it travels as a whole, so it cannot arrive half-connected. Wire it from 'CV Tuple' or type the components in place.
alphaFLOAT0.0000-1e+38–1e+38Free scaling parameter between 0 (when all the pixels in the undistorted image are valid) and 1 (when all the source image pixels are retained in the undistorted image). See #stereoRectify for details.
newImgSizeoptCV_TUPLE0,0Image size after rectification. By default, it is set to imageSize . One value with 2 components (w, h) - it travels as a whole, so it cannot arrive half-connected. Wire it from 'CV Tuple' or type the components in place.
centerPrincipalPointoptBOOLEANfalseOptional flag that indicates whether in the new camera intrinsic matrix the principal point should be at the image center or not. By default, the principal point is chosen to best fit a subset of the source image (determined by alpha) to the corrected image. Preset to the OpenCV default (False).

Outputs (2)

NameTypeDescription
newCameraMatrixNPARRAY—
validPixROIBOUNDING_BOXnew_camera_matrix Output new camera intrinsic matrix. The function computes and returns the optimal new camera intrinsic matrix based on the free scaling parameter. By varying this parameter, you may retrieve only sensible pixels alpha=0 , keep all the original image pixels if there is valuable information in the corners alpha=1 , or get something in between. When alpha>0 , the undistorted result is likely to have some black pixels corresponding to "virtual" pixels outside of the captured distorted image. The original camera intrinsic matrix, distortion coefficients, the computed new camera intrinsic matrix, and newImageSize should be passed to #initUndistortRectifyMap to produce the maps for #remap . A cv2 Rect as core BOUNDING_BOX data ({x, y, width, height}, nested one group per frame) - feed 'Crop By Bounding Boxes', 'Draw BBoxes', or 'CV Split Tuple' for x/y/w/h.