Nodes/ComfyUI CV/cv2.initUndistortRectifyMap
ComfyUI Node

cv2.initUndistortRectifyMap

Build the lookup table once, fix the whole video with it

By bmad4ever·Created 3 months ago·Updated 14 days ago· 1
cv2.initUndistortRectifyMap
  • cameraMatrix
  • distCoeffs
  • R
  • newCameraMatrix
  • size
  • map1
  • map2
◄m1typeCV_16SC2 (fixed-point, faster remap)►

Straight lines come out curved on a phone or a GoPro, and every "lens correction" button in the world is doing the same thing underneath: computing a new coordinate for each output pixel. cv2.undistort redoes that math on every call. initUndistortRectifyMap does it once, hands you two arrays, and then cv2.remap just samples through them.

That split is the whole reason this node exists. If you're correcting one photo, cv2_undistort is one node and you're done. If you're correcting a 300-frame clip, you build the map once and reuse it for every frame - same numbers, same sampling path, and you can see the map itself when something looks wrong.

What the two maps are

map1 and map2 encode, for each pixel of the output, which source coordinate to read and how. That's the remap contract: given a destination pixel, where do I sample from? Both come out as NPARRAY and go straight into cv2_remap, which in the pack's own stereo workflows is wired exactly that way - map node → Remap left (rectified).

Inputs you actually set

  • cameraMatrix - the 3×3 intrinsics K. CV Camera Matrix builds an approximate one from fx/fy/cx/cy if you don't have a calibration; CV Calibrate Camera (Chessboard) or the circle-grid / ChArUco / fisheye variants produce a real one from views. NPARRAY only - it's data, not a picture.
  • distCoeffs - (k1, k2, p1, p2[, k3[, k4, k5, k6[, s1, s2, s3, s4[, τx, τy]]]]), 4/5/8/12 or 14 elements. Also NPARRAY only.
  • R - the object-space rectification rotation. For plain mono undistortion an identity (CV Eye, 3×3) is correct; for stereo, wire R1/R2 from cv2_stereoRectify.
  • newCameraMatrix - the camera matrix the output is framed with. Use K, or the output of cv2_getOptimalNewCameraMatrix whose alpha trades "crop away the invalid black edges" against "keep every original pixel", or stereoRectify's P1/P2.
  • size - the undistorted image size, one CV_TUPLE value, so it arrives whole. Wire it from CV Array Size (reads width/height off a real image) or type it.
  • m1type - CV_16SC2 (fixed-point, faster remap) by default. That's the fast path: int16 packed coordinates plus a coefficient map, roughly 30% quicker in remap. CV_32FC1 (float32, standard) is the boring float version - the pack's shipped stereo workflows use this one, so if a downstream node objects to a 2-channel int16 array, switch.

Outputs: map1 and map2, both NPARRAY.

Wiring it end to end

The pack's calibration workflows are the reference. In 52_circle_grid_calibration.json, each of the two map nodes is fed from CV Stereo Calibrate (Circle Grid) for cameraMatrix and distCoeffs, cv2_stereoRectify for R and newCameraMatrix, and a size node for size - then cv2_remap does the actual work. Stereo pointcloud, AR occlusion and the phone-camera exercises use the same pattern. In mono, swap the stereo calibration for CV Calibrate Camera (Chessboard) and feed an identity into R.

Installing the pack

Manager → search comfyui_cv → install, restart. By hand:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Python ≥ 3.12 and a ComfyUI built on the V3 node API. Behaviour is curated against OpenCV 5.0.0.93; other versions "may behave differently", which is the README's polite way of saying your mileage varies. Nothing to download for this node - the sample calibration images live in the repo's example_inputs/, and there's a one-node workflow (01_install_example_inputs.json) that copies them into ComfyUI/input for you if you want to try the exercises.

Where people get burned

Resolution drift. The map is baked for one image size. Re-render at 1024 instead of 1280 and your correction is wrong in a way that looks almost right. Rebuild the map.

The two-map chicken-and-egg. size must match the image you're going to remap. Read it from the image itself with CV Array Size rather than typing 1920×1080 from memory.

Black borders, then surprise. Undistorting with a new camera matrix that keeps every source pixel gives you valid output and warped edges; getOptimalNewCameraMatrix's alpha is the dial. Set it deliberately.

Stacking distortion twice. If your loader already applied lens correction (a phone's processed JPEG, some footage pipelines), running the map again doubles the warp in the opposite direction. Check on a straight edge before committing to a batch.

This is a hand-curated wrapper around a well-documented OpenCV function, but the pack around it is a personal, LLM-assisted project whose README warns against production use without your own review. The maths is OpenCV's; the plumbing is worth a glance.

Categoryimage/CV/low-level/cv2 I

Inputs (6)

NameTypeDefaultDescription
cameraMatrixNPARRAYInput camera matrix $A=\vecthreethree{f_x}{0}{c_x}{0}{f_y}{c_y}{0}{0}{1}$ . A data array (points / matrix), NOT an image - only an NPARRAY link is accepted here.
distCoeffsNPARRAYInput vector of distortion coefficients $(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6[, s_1, s_2, s_3, s_4[, \tau_x, \tau_y]]]])$ of 4, 5, 8, 12 or 14 elements. If the vector is NULL/empty, the zero distortion coefficients are assumed. A data array (points / matrix), NOT an image - only an NPARRAY link is accepted here.
RNPARRAYOptional rectification transformation in the object space (3x3 matrix). R1 or R2 , computed by #stereoRectify can be passed here. If the matrix is empty, the identity transformation is assumed. In #initUndistortRectifyMap R assumed to be an identity matrix. A data array (points / matrix), NOT an image - only an NPARRAY link is accepted here.
newCameraMatrixNPARRAYNew camera matrix $A'=\vecthreethree{f_x'}{0}{c_x'}{0}{f_y'}{c_y'}{0}{0}{1}$. A data array (points / matrix), NOT an image - only an NPARRAY link is accepted here.
sizeCV_TUPLE0,0Undistorted image size. One value with 2 components (w, h) - it travels as a whole, so it cannot arrive half-connected. Wire it from 'CV Tuple' or type the components in place.
m1typeCOMBOCV_16SC2 (fixed-point, faster remap)Type of the first output map that can be CV_32FC1, CV_32FC2 or CV_16SC2, see #convertMaps

Outputs (2)

NameTypeDescription
map1NPARRAY—
map2NPARRAY—