cv2.fisheye.distortPoints (1/2)

Undistortion is the direction everyone learns first: you have a wide-angle photo, you flatten it, done. cv2.fisheye.distortPoints is the other direction - taking points that live in straightened, ideal coordinates and telling you where they'd land in the distorted image. Once you notice that direction exists, a whole class of "why doesn't my overlay line up?" problems gets an answer.

Concrete uses. You've straightened a fisheye frame and drawn something on the straight version (a grid, a mask outline, a synthetic object, an annotation) and now you want it back on the original photo, warped the way the lens would have warped it. Or you're verifying a calibration: undistort a set of points, distort them back, and see whether you return to where you started. Or you're building test data with known ground truth to check your own undistortion pipeline.

Inputs

undistorted is the point array - Nx1x2 float coordinates in the undistorted frame. K is the 3×3 camera matrix and D is the fisheye distortion vector. The socket type note matters here: this is a submodule function, and the pack's generator gave its matrix parameters the generic image-ish socket, so K and D will happily accept an IMAGE link. Don't. They're data arrays; wire them from CV Camera Matrix, Parse Matrix, or the camera_matrix/dist_coeffs outputs of CV Fisheye Calibrate (Chessboard). A picture arriving where a 3×3 matrix belongs fails inside cv2 with a message you'll have to decode.

And the D that belongs here has four coefficients (k1..k4), because this is the equidistant fisheye model - not the five-coefficient Brown-Conrady vector the pinhole calibrator produces. The two are not interchangeable; feeding pinhole coefficients into a fisheye function gives a plausible-looking but wrong warp, which is the worst kind of wrong.

alpha is an optional free-text literal: leave it blank and the parameter isn't passed at all, meaning OpenCV's own default applies. Type 0.0 if you want it explicit. Same convention as every optional scalar in this pack's generated wrappers.

Output is a single nparray - the distorted coordinates, same layout, ready for cv2_perspectiveTransform-free direct use, for CV DrawPoints to eyeball over the real photo, or for whatever geometry you're compositing.

The frame convention trap

OpenCV's documentation for this function is terse, and the parameter naming - undistorted, K - leaves the question open whether the points are expected in normalized or pixel coordinates. That ambiguity is the thing that will cost you an afternoon. The cheap insurance is a round trip: take a known point set, run it through cv2_fisheye_undistortPoints and then through this node, and check you land back on the original coordinates. If you don't, the two calls are speaking different frames, and the usual culprit is that the undistort step was handed a new camera matrix (P) while the distort step was handed the original K.

Do that once with your camera and you'll never have to guess again.

Where it sits in a wider graph

The fisheye family in this pack exists to serve the wide-angle pipeline: CV Fisheye Calibrate (Chessboard) gives you K and the four coefficients from a batch of board views, CV Fisheye Undistort straightens images with them, and cv2_fisheye_projectPoints handles 3D-to-pixel work. distortPoints is the bridge for any points you've produced in the straightened frame - including annotations from a 3D preview or a purely synthetic overlay - so they can be composited back over the raw footage and actually track the lens.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

ComfyUI Manager: search "ComfyUI CV", install, restart. Python ≥ 3.12 and a recent ComfyUI on the V3 node API. The contrib wheel is not optional here - the fisheye module lives in the contrib distribution, and if some other pack installs a plain opencv-python over it, the whole cv2.fisheye namespace goes away. That's a routine ComfyUI dependency collision rather than a bug in this pack; tools/repair_opencv_contrib.py --check reports it and --apply repairs it.

Common issues

cv2 complains about D. It must be four elements, as a column vector or a 4-element array. Five pinhole coefficients, or the 14-element rational model, are rejected or misinterpreted.

Points land somewhere inexplicable. Check the camera matrix you used matches the image you're comparing against - same resolution, same crop. Distortion is resolution-specific, and a K from a 1920×1080 calibration applied to a 960×540 frame is off by a factor of two in focal length.

Nothing visible before you preview. Output is an NPARRAY, which has no viewer of its own. Preview CV Array renders arrays directly, and Inspect CV Data prints shape, dtype and value statistics - the two nodes you'll use constantly in this pack.