cv2.fisheye.distortPoints (2/2)

Two cv2.fisheye.distortPoints nodes, one extra input between them. The 1/2 version asks for one camera matrix. This 2/2 version asks for two: Kundistorted and K. That's the whole difference, and it's the input that makes the node actually useful when you're moving between two different camera frames.

Here's the scenario it exists for. You undistorted a wide-angle photo by estimating a new camera matrix - the pack's CV Fisheye Undistort node does this, with its "estimate new K" option, and the raw route is cv2.fisheye.estimateNewCameraMatrixForUndistortRectify feeding cv2.fisheye.initUndistortRectifyMap. Your straightened image is now expressed in that new intrinsics, which are not the original K: a different focal length, usually a shifted principal point, a different field of view. If you draw something in this straightened frame, its coordinates belong to Kundistorted. To composite that drawing back onto the raw photo, you need to map it from Kundistorted into K, through the distortion model.

That's this node. Input: points in the undistorted frame plus its intrinsics. Output: where those points sit in the distorted image.

Inputs

undistorted - the point array, Nx1x2 floats. Kundistorted - the 3×3 camera matrix that the points are already expressed in, i.e. the one the undistorted image was built with, not the original. K - the 3×3 matrix of the distorted target image, which is normally the camera's true intrinsics from calibration. D - the fisheye distortion coefficients: four, k1 through k4, the equidistant model. Same warning as everywhere in this module: a pinhole five-coefficient vector does not belong here, and it will produce a warp that looks reasonable and is wrong.

alpha is an optional free-text literal. Blank means the parameter isn't passed and OpenCV's default applies. That's the pack's convention for optional scalars on generated wrappers: an untyped field that accepts a Python literal (0.0, 1.5), where blank genuinely means "omit".

The output is one nparray of distorted coordinates - pixel positions in the original image, ready for an overlay drawn onto the un-straightened photo.

Why bother going backwards

Three reasons. Composite honesty: an annotation drawn in the straightened frame and pasted as-is over the original will float away from the feature it's labelling, because the lens bent everything between them. Mapping the geometry back through Kundistorted → K fixes that properly rather than by nudging. Overlay verification: if you're checking a rectification or an AR alignment on raw footage, you want to see the marker land where it should in the frame the camera actually recorded. Reprojection plumbing: geometry that comes out of a 3D or rectified stage and needs to be compared with measured pixels is exactly this conversion.

If the two matrices are identical - you didn't estimate a new K, you used the original - then this node and the 1/2 version produce the same answer, and you can use the simpler one. The pair exists because the interesting case has two frames.

Practical notes

Round-trip your calibration once before trusting anything: a point set through cv2_fisheye_undistortPoints and back through this node should return to where it started. If it doesn't, you're mixing up which matrix belongs to which frame - the single most common mistake with this family of calls.

Distortion coefficients and camera matrices are resolution-specific. A K built for a 4K capture applied to a downscaled frame is wrong by the scale factor in focal length, and the resulting points will be systematically off in a way that looks like a calibration error.

And remember the outputs are NPARRAYs with no viewer: Preview CV Array to look, Inspect CV Data to read shapes and numbers.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

ComfyUI Manager: search "ComfyUI CV", install, restart. Python ≥ 3.12 and a V3-API ComfyUI. cv2.fisheye comes from the contrib wheel, so this is one of the nodes that disappears if a non-contrib OpenCV build ends up in your environment - all OpenCV Python wheels share one site-packages/cv2, so installing a plain opencv-python over the contrib one empties the submodules. tools/repair_opencv_contrib.py --check / --apply in the pack directory handles that.

Common issues

Points come back shifted consistently. Wrong K or Kundistorted for the resolution, or balance differing between the matrix you estimated and the image you're comparing against.

cv2 raises on the arrays. Four coefficients in D; 3×3 float matrices for both K inputs. Inspect CV Data first - it prints shapes and dtypes, which is faster than reading a stack trace.

Coordinates look plausible but the overlay is mirrored. Check whether the image path you're compositing onto was flipped or transposed anywhere (a preview, a cv2_flip, a mask conversion). Flipping the image and not the points is a classic.