cv2.fisheye.initUndistortRectifyMap

This is the engine room of fisheye undistortion. cv2.fisheye.initUndistortRectifyMap doesn't warp an image - it precomputes the mapping tables that say, for every pixel of the output, where in the input to sample from. You then hand those tables to cv2_remap and it does the lookup. Two nodes instead of one, and in exchange you get the thing that makes video work: the maps don't change between frames, so you build them once and reuse them.

The curated CV Fisheye Undistort node does exactly this pair internally, in one box. Reach for the raw wrappers when you want to see the maps, control the output size separately from the input, or reverse the maps for a projection you're building yourself.

Inputs

K is the 3×3 camera matrix and D the fisheye distortion vector - four coefficients (k1..k4). Both come from CV Fisheye Calibrate (Chessboard). This is the equidistant model; a pinhole five-coefficient vector from the Brown-Conrady calibrator will produce a smooth, plausible, wrong unwarp, so keep the two families apart.

R is the 3×3 rectification rotation. Identity for an ordinary undistort; from cv2_fisheye_stereoRectify when you're rectifying a stereo pair. P is the new camera matrix - the framing decision. Pass the output of cv2.fisheye.estimateNewCameraMatrixForUndistortRectify if you want the estimated field of view; pass the original K if you want to keep the original focal length and simply crop away everything that straightening pushes off-frame.

size is the output size, a two-component Size value authored with CV Tuple or typed in place - (width, height). m1type is a dropdown with two options: CV_16SC2 (fixed-point, faster remap), which is the default, and CV_32FC1 (float32, standard). Faster versus boring. If you're remapping video frames, fixed-point is real speed; if anything downstream complains about map types, switch to float32 and stop thinking about it.

One typing note worth internalising: this is a submodule function, so its matrix inputs got the generic image-ish socket from the pack's generator. K, D, R and P will accept an IMAGE link. They're data arrays. Wire them from calibration or Parse Matrix, never from a picture.

Outputs

Two NPARRAYs, in cv2's return order: the first is map1 (the x lookup) and the second is map2 (the y lookup). They come out with the generic socket labels nparray_0 and nparray_1 - that's what the generated wrappers do for submodule functions; the top-level cv2_initUndistortRectifyMap in the same pack labels its two outputs map1 and map2, and they're the same thing. If you're ever unsure which is which, the order is cv2's, and CV Split Tuple-style splitting isn't needed: just wire both, in order, into remap.

Wiring it up

cv2_remap takes the image, map1 and map2, a dsize (set it to the same size you gave the map builder) and an interpolation flag - INTER_LINEAR for a smooth, normal-looking result. Mismatch the dsize and the map size and you'll get a cropped or padded output, which is the single most common confusion with this pair. remap is in the pack's per-frame-safe set, so an IMAGE batch is looped frame by frame and re-stacked: build the maps once, then remap a whole video batch without recomputing anything.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

ComfyUI Manager: search "ComfyUI CV", install, restart. Python ≥ 3.12 and a ComfyUI on the V3 node API. cv2.fisheye lives in the contrib wheel, so a plain opencv-python installed over it by another pack empties the contrib submodules and this node disappears - a normal ComfyUI collision, not a fault in the pack. tools/repair_opencv_contrib.py --check then --apply.

Common issues

The new K is all NaNs. A known OpenCV wart when the fisheye calibration left k3/k4 unconstrained. Recalibrate with CALIB_FIX_K3 | CALIB_FIX_K4 - the pack's 53_fisheye_calibration.json walks through it - or use the curated undistort node, which detects the case and falls back to the original K instead of handing you a broken map.

Maps built at one resolution, applied at another. Distortion and focal length are in pixels. A map table for 4K is meaningless on a 1080p frame; rebuild the maps per resolution.

Black wedges or absent corners in the result. That's the P you fed it, not a bug. Original K crops to the sensor's valid area; an estimated K with a high balance keeps more scene and more empty space. Look at both framings before choosing.

Weird smearing at the frame edges. Fisheye straightening stretches the extreme periphery a lot, and there's simply less data per output pixel out there. INTER_LINEAR is the sane default; the edges will still be soft.