CV Fisheye Undistort

Straight lines should be straight. This node takes a wide-angle or fisheye image and un-bends it, using OpenCV's equidistant fisheye model - the same one CV Fisheye Calibrate fits - and it does the one thing most "lens correction" filters don't: it lets you choose what happens to the frame.

It's part of bmad4ever's ComfyUI CV pack (bmad4ever/comfyui_cv, forked from Gerold Meisinger's opencv-comfyui) - ~470 auto-generated raw cv2.* wrappers plus curated nodes for the parts a raw wrapper can't express. cv2.fisheye.initUndistortRectifyMap plus cv2.remap is exactly that kind of part: two calls, a camera-matrix decision and a NaN trap in the middle.

The inputs, and why there are only three that matter

image takes a ComfyUI IMAGE/MASK directly or an NPARRAY. camera_matrix is the 3×3 K from the calibrate node, and dist_coeffs must be the four fisheye coefficients. Feed it a five-element pinhole vector and it raises, with a message that says where the vector came from - deliberately, because silently truncating it would produce a wrong image that looks fine.

Then new_camera, which is the framing decision:

• "same K" - keep the original focal length. Anything the straightening pushes outside the frame is gone. This is cv2's own default behaviour and the tight, crisp-looking option.
• "estimate new K" - recompute the intrinsics from the undistorted image bounds. Now balance (default 0, 0–1) matters: it blends between the two focal lengths cv2 derives, 0 = the longer one, 1 = the shorter (wider view, everything smaller). fov_scale (advanced, 1.0) is an extra divisor on the estimated focal length if you want to zoom out past either end.

Measured on the pack's shipped example, the two ends are wildly different: fx 166 at balance 0.0 versus fx 50 at balance 1.0, with 3.7% of the frame left black at 0.0 and 0.3% at 1.0. And the honest note in the docs: which end leaves black corners is lens-dependent, so look at both instead of assuming. That's the sort of thing a tutorial would fudge, and the pack doesn't.

Outputs

image - the straightened frame, same size as the input.

new_camera_matrix - the 3×3 K that the output image is expressed in. This is the output people miss. Anything downstream that projects into the undistorted image (solving a pose, drawing a 3D overlay, unprojecting a depth map) needs this K, not the original one. Feeding the distorted camera's K to a projection into a straightened image is a subtle, plausible-looking error - the same class of mistake as the coefficient mix-up above.

How it goes about it

initUndistortRectifyMap builds one (map_x, map_y) pair for the whole frame - every output pixel stores which input coordinate to sample - and remap does the resample in one pass. That's why this is fast and deterministic: no search, no iteration, just a precomputed lookup. It's also why the operation is not invertible by re-running it with different numbers; if you need the distorted frame back, keep it.

If you're building a pipeline rather than fixing one image, note that the pack's CV Flow Map and its remap-family nodes compose several distortions into a single cv2_remap call - undistortion can be one stage in that chain instead of a separate resample that softens the image twice.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Or find ComfyUI CV in ComfyUI Manager. Requires Python ≥ 3.12 and a recent ComfyUI on the V3 node API - an older build fails at import, not with a broken node. For the bundled demo, run workflows/01_install_example_inputs.json once, then reload the page so the Load Image dropdowns pick up the copied fisheye boards. Then workflows/53_fisheye_calibration.json shows same-K, estimated-K at both balances, and a no-board case, side by side.

Common issues

Black wedges in the corners. Expected with "same K" - you threw away the pixels that were never there. Switch to estimated K and nudge balance until the frame is filled, accepting that the scene gets smaller.

The result looks smeared toward the edges. That's real: undistorting stretches the periphery, so the pixels near the frame edge are being magnified. Nothing to tune - it's why the framing choice exists.

It falls back to the original K and logs a warning. The calibrate step left k3/k4 unconstrained, cv2's estimator returned all-NaN, and the node refused to hand you a NaN camera. Re-calibrate with CALIB_FIX_K3 | CALIB_FIX_K4 in CV Fisheye Calibrate's flags.

A pinhole calibration's coefficients. They're a different model with a different meaning. It won't error; it'll just be wrong. Use the fisheye calibrate node.