cv2.fisheye.solvePnPRansac

Plain PnP is only as good as its worst correspondence. One mismatched point out of twenty drags the solution off, and because the solver still converges, nothing tells you. cv2.fisheye.solvePnPRansac is the same fisheye pose problem with RANSAC wrapped around it: it repeatedly solves from random minimal subsets, counts how many of your correspondences land within a reprojection tolerance of each candidate pose, and refines the winner using only those inliers. If your 2D points came from a detector, a keypoint matcher, or your own clicking, this is the node you reach for - the plain cv2.fisheye.solvePnP is for clean, trusted correspondences.

Both live in ComfyUI CV (bmad4ever), the pack that exposes OpenCV 5.0 as ComfyUI nodes (roughly 470 auto-generated raw cv2.* wrappers, plus a curated layer on top). The raw ones are deliberately thin: OpenCV's signature, OpenCV's defaults, NPARRAY sockets.

How it works

RANSAC is a loop with a probability argument. Each iteration samples the minimum number of correspondences a fisheye PnP needs, solves, and counts inliers - points whose reprojection error is under the tolerance. confidence (0.99 here, a preset FLOAT rather than a blank field) is the probability the loop will find a good sample; iterationsCount caps the work. OpenCV's defaults, which you get by leaving those fields blank, are 100 iterations and an 8-pixel reprojection error - for a wide lens at the edge of frame, 8 px is generous and will happily accept a sloppy pose, so if your result looks "found but wrong", tighten reprojectionError before you blame the calibration.

The arithmetic between confidence, reprojectionError and the inlier ratio is worth internalising: at a 50% outlier rate, 99% confidence needs a lot more iterations than at 5%, because the probability of drawing an all-inlier minimal set collapses fast with the outlier fraction.

Inputs and outputs

Inputs are the pose problem itself: objectPoints (Nx3, model space), imagePoints (the matched pixels), cameraMatrix (3×3 K), distCoeffs (four fisheye coefficients k1..k4 - a five-element pinhole vector is the classic wrong answer that doesn't raise), then optional useExtrinsicGuess, iterationsCount, reprojectionError, flags as blank-means-default strings, confidence, and the usual criteria_type/criteria_max_count/criteria_epsilon trip.

Outputs are bool, then three NPARRAYs - and the third one is why this node is worth the extra socket:

• nparray_1 - rvec, the rotation vector.
• nparray_2 - tvec, the translation.
• nparray_3 - the inlier mask. Wire it into CV Filter Points By Mask to keep only the correspondences the solver actually believed, or draw them in one colour and the rejects in another. This is the diagnostic that plain PnP doesn't give you, and it's the fastest way to find the point you clicked on the wrong corner.

The bool is not a confidence score - it's "did RANSAC end up with a solution at all". Combine it with the inlier count for anything you'd automate.

One footnote on useExtrinsicGuess: cv2's binding takes rvec/tvec as optional in/out arguments, and this wrapper declares them as returns rather than inputs, so there is nowhere to supply a starting pose. Leave it blank.

Install

ComfyUI Manager → ComfyUI CV, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install -r comfyui_cv/requirements.txt

Python ≥3.12, current ComfyUI (V3 node API), restart after. That pulls opencv-contrib-python-headless~=5.0.0.93; the fisheye module is a contrib submodule, so a non-contrib OpenCV wheel will make this whole category vanish - python ComfyUI/custom_nodes/comfyui_cv/tools/repair_opencv_contrib.py --check diagnoses it, --apply fixes it (server stopped).

Common issues

found=false with an empty inlier mask. Fewer correspondences than the minimal solver needs, or every candidate pose failing the tolerance. Print the point counts first - a feature matcher that returned six pairs on a textureless wall is the usual story.

It finds a pose, and the pose is a flip. With four coplanar points, planar PnP has two geometrically valid answers. RANSAC picks the one with more inliers, which is not always the physically correct one. The pack's PnP playground workflow exists to make that ambiguity visible rather than hidden.

Consistently huge inlier counts on nonsense data. reprojectionError is too loose. Tighten it; if the inlier count then collapses, your K/D is the problem, not the points.

Enable the pack's example inputs if you want test data to play with. Load workflows/01_install_example_inputs.json, press Run, then reload the ComfyUI page so the Load Image/Load Video dropdowns rebuild with the copied files in ComfyUI/input.