cv2.fisheye.solvePnP

This is the pose solver for wide-angle cameras. You give it 3D points in the object's own frame and the matching 2D pixels where you found them, plus the lens's intrinsics, and it hands back the camera's rotation and translation in that object frame. It's the same job cv2.solvePnP does with a pinhole model - but it runs the fisheye projection inside the optimisation, which is the difference between an overlay that fits and one that fits in the middle of the frame and bends at the edges.

It's part of ComfyUI CV (bmad4ever), a pack that exposes OpenCV 5.0 as ComfyUI nodes: about 470 auto-generated raw cv2.* wrappers plus curated high-level nodes. These low-level ones are uncurated by intent - the README says so - so think of this node as "OpenCV's signature with sockets", not as a guided wizard.

How it works

OpenCV solves this iteratively: start from a guess pose, project the 3D points through the fisheye model, measure the pixel error, and refine with Levenberg–Marquardt until the error stops improving. The three criteria_* fields on the node are exactly that loop's stop condition - max iterations (default 30), epsilon (default 0.001), or whichever hits first. Leave them; they're fine.

The outputs are the interesting part. Three sockets:

• bool - whether a solution was found at all.
• nparray_1 - the rotation vector (rvec).
• nparray_2 - the translation vector (tvec).

Those two are why you're here. rvec/tvec are the input of cv2.fisheye.projectPoints, of cv2.drawFrameAxes, and of the whole family of "put the 3D thing on the 2D frame" nodes. Note the socket names: the wrapper names multi-return outputs by index, so nparray_1/nparray_2 mean "second and third thing cv2 returned", in cv2's order - (retval, rvec, tvec).

The inputs that matter

• objectPoints - the 3D model-space points, Nx3, float. For a chessboard, CV Grid Points exists precisely because you can't type a 9×6 board's 54×3 values by hand. Careful with the fisheye convention: OpenCV's fisheye family wants (1, N, 3), and the curated CV Fisheye Calibrate node comments at length about that shape trapping people.
• imagePoints - where those same points landed in the picture. From corner detection, from cv2.findChessboardCorners, from CV Annotate Points if you're clicking by hand, or from a feature-matching pipeline.
• cameraMatrix / distCoeffs - K and, critically, four distortion coefficients k1..k4. Five-element pinhole coefficients don't error here; they produce a wrong-but-plausible pose, which is worse.
• useExtrinsicGuess / flags - both are advanced blank-means-default string fields. Here's the honest advice: leave useExtrinsicGuess blank. cv2's Python binding takes rvec/tvec as optional in/out arguments, and this wrapper drops them from the inputs and returns them instead, so there's no socket to hand it the starting pose it would be guessing from. Setting it true would have cv2 refine from whatever it initialises internally, which is not what you meant.
• flags - a raw int here, so it's the SOLVEPNP_* family typed as a number rather than a dropdown. Blank gives OpenCV's default (iterative). Most people never touch it.

Install

ComfyUI Manager → search ComfyUI CV, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install -r comfyui_cv/requirements.txt

Python ≥3.12 and a recent ComfyUI (V3 node API). Restart ComfyUI afterwards.

Common issues

Fewer than four correspondences, or coplanar weirdness. PnP is not magic: with too few points cv2 has nothing to solve, and a planar target has a genuine two-fold ambiguity (the classic planar flip - the pack's own PnP playground workflow demonstrates it by solving the same board with three different solvers, so you can see the runner-up pose is a genuine second answer rather than an artefact - and that only the reprojection error tells you which is physical).

A garbage pose with no warning, RMS included. This is the fisheye counterpart of a well-known calibration trap: CALIB_RECOMPUTE_EXTRINSIC is not optional in cv2.fisheye.calibrate, and the pack's example workflow measures the same eight boards calibrating to 0.13 px RMS with it and 22 px without. A bad K/D doesn't make solvePnP fail; it makes it return a confident wrong answer. Sanity-check by projecting back and looking at the overlay.

The category is empty after install. You've got a non-contrib OpenCV wheel on top of the contrib one - they share one site-packages/cv2, so contrib submodules go quietly empty. Run python ComfyUI/custom_nodes/comfyui_cv/tools/repair_opencv_contrib.py --check and then --apply with the server stopped.

If your 2D points are noisy, don't use this node. Use cv2.fisheye.solvePnPRansac and let it throw out the outliers - that's the whole reason the RANSAC variant exists.