OpenCV solvePnPRefineLM_0

solvePnP_0 gets you a pose. solvePnPRansac_0 gets you a robust pose. solvePnPRefineLM_0 is what you do after - a nonlinear refinement pass that takes a good-but-not-perfect estimate and squeezes the last bit of accuracy out of it using the Levenberg-Marquardt optimizer. In OpenCV's normal pipeline, you run solvePnP first and then hand its rvec/tvec to solvePnPRefineLM to fine-tune. This node is the second half of that pair.

When does that matter in a ComfyUI workflow? When your pose output is feeding something pixel-sensitive - a warp, a projection, a compositing step - and the difference between "close" and "locked" shows up as a few pixels of drift. Refinement is cheap and it's the difference between a pose you can use and a pose you can trust.

The inputs

Same point-correspondence setup as the rest of the family:

• objectPoints (NPARRAY) - 3D points, N×3.
• imagePoints (NPARRAY) - 2D projections, N×2.
• cameraMatrix (NPARRAY) - 3×3 intrinsics.
• distCoeffs (NPARRAY) - distortion coefficients.
• rvec (NPARRAY) - the initial rotation estimate. Required here, unlike in solvePnP_0 - this node refines, it doesn't solve from scratch.
• tvec (NPARRAY) - the initial translation estimate.

And then there's criteria (STRING), which is where this pack's quirks show up. criteria is a TermCriteria in OpenCV - a composite type - so the node exposes it as a string and parses it with Python's ast.literal_eval. You type a tuple literal:

(3, 100, 1e-6)

That's (type, maxCount, epsilon): type 3 means TermCriteria::COUNT + EPS (stop after 100 iterations or when the improvement drops below 1e-6), 1 is count-only, 2 is epsilon-only. If you get invalid syntax (<unknown>, line 0), this string is what's malformed.

Outputs: nparray_0 (refined rvec) and nparray_1 (refined tvec). No success flag - this node assumes you handed it a workable pose and just improves it. Garbage in, slightly-polished garbage out, so do the sanity check on solvePnP's bool first.

How to use it in a graph

Feed it the rvec/tvec outputs of solvePnP_0 or solvePnPRansac_0 (their nparray_1 / nparray_2), keep the same objectPoints/imagePoints/cameraMatrix/distCoeffs, wire the result to whatever consumes a pose. All nparray plumbing - Image2Nparray in, Nparrays2Image out, batch size 1.

Install

Standard for the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/geroldmeisinger/opencv-comfyui
pip install opencv-contrib-python

or ComfyUI Manager → "opencv-comfyui". Restart. Deps: opencv-contrib-python, numpy, torch - nothing to download.

Troubleshooting

• invalid syntax (<unknown>, line 0) - the criteria literal is malformed. It needs to be a valid Python tuple like (3, 100, 1e-6), matching the README's "use literal strings for composite parameters."
• Pose barely moves - that's normal; refinement is for small corrections. If it moves a lot, your initial estimate was bad.
• guidedFilter import error at startup - conflicting OpenCV packages; README has the fix.

It's the understated workhorse of the pose-estimation family: no drama, no success flag, just a better answer than you fed it.