OpenCV solvePnPRefineVVS_0

solvePnPRefineLM_0 refines a pose with Levenberg-Marquardt. solvePnPRefineVVS_0 is the same job done a different way - Virtual Visual Servoing, a control-theory approach that iteratively steers a virtual camera until the projected points line up with your 2D observations. It's the second refinement option OpenCV offers for pose estimation, and the only practical difference for you is one extra input: VVSlambda.

Both refiners do the same thing at the workflow level: take a decent initial rvec/tvec and make it more accurate. So why pick VVS over LM? Mostly taste and edge cases. VVS is sometimes more stable when your initial guess is off or the geometry is unpleasant, and the VVSlambda gain gives you a tuning knob LM doesn't expose. In practice, most people reach for whichever their tutorial showed, and both are fine.

The inputs

• objectPoints (NPARRAY) - 3D points, N×3.
• imagePoints (NPARRAY) - 2D projections, N×2.
• cameraMatrix (NPARRAY) - 3×3 intrinsics.
• distCoeffs (NPARRAY) - distortion coefficients.
• rvec (NPARRAY) - initial rotation estimate.
• tvec (NPARRAY) - initial translation estimate.
• criteria (STRING) - the TermCriteria as a tuple literal, parsed with ast.literal_eval. (3, 100, 1e-6) = stop after 100 iterations or when the change drops below 1e-6.
• VVSlambda (FLOAT) - the servoing gain. This controls how aggressively each iteration steers the virtual camera. Too big and the refinement overshoots and oscillates; too small and it crawls. The OpenCV default is around 1, and 0.5–2 is a sane search range.

Outputs: nparray_0 (refined rvec), nparray_1 (refined tvec). No success flag - like RefineLM, this assumes you fed it a workable pose.

Where the quirks are

The criteria string is the same trap as its LM sibling: it must be a valid Python tuple literal or you'll see the pack's invalid syntax (<unknown>, line 0) error. And VVSlambda is a genuine "feel" parameter - you can't derive the right value from the docs, you nudge it until refinement stops oscillating. If your refined pose jumps around frame to frame, lower the lambda before you blame the node.

Wiring is pack-standard: nparray in, nparray out. Feed it the rvec/tvec from solvePnP_0 or solvePnPRansac_0, use Image2Nparray / Nparrays2Image at the edges, batch size 1.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/geroldmeisinger/opencv-comfyui
pip install opencv-contrib-python

or ComfyUI Manager → "opencv-comfyui". Restart. Only opencv-contrib-python (+ numpy, torch) matters - no model files.

Troubleshooting

• invalid syntax (<unknown>, line 0) → the criteria literal is malformed; needs a real tuple like (3, 100, 1e-6).
• Refined pose oscillates or explodes → VVSlambda is too high; dial it back toward 0.5.
• Refined pose barely changes → either you started close (fine) or the initial pose was bad (go fix upstream).
• Cannot import name 'guidedFilter' at startup → conflicting OpenCV packages; README links the fix.

It's the less-traveled refinement path with one more knob to turn. If LM isn't converging for your geometry, this is the one to try.