OpenCV projectPoints_0

projectPoints_0 is the node that answers a camera-geometry question: given a set of 3D points in the world, where would they land on a 2D image taken by a camera with known position, orientation, and lens? It's cv2.projectPoints, and it's the mathematical heart of pose estimation, augmented reality, and camera calibration - ported into ComfyUI by the opencv-comfyui pack's auto-generator.

Let me set expectations before we go further: this is the most "computer vision textbook" node in the pack, and the least "AI image generation" node you'll ever add to a graph. It needs a calibrated camera - intrinsics, extrinsics, distortion - none of which a text-to-image workflow hands you. If you're generating pretty pictures, you don't need this. If you're doing real-image vision work inside ComfyUI, it's the real deal.

How it works

The projection is the standard pinhole-camera model. You provide:

• objectPoints (NPARRAY) - the 3D points, shape (N, 3) or (N, 1, 3), float32.
• rvec, tvec (NPARRAY) - the camera's rotation vector and translation vector relative to the world. Together they define where the camera is and which way it's pointing.
• cameraMatrix (NPARRAY) - the 3×3 intrinsics: focal length, principal point. This is the lens's fingerprint.
• distCoeffs (NPARRAY) - lens distortion coefficients. Can be empty if your lens is (theoretically) perfect.
• aspectRatio (FLOAT) - a newer OpenCV parameter for the pixel aspect ratio used in undistortion. Here it's a required input; 1.0 is the neutral value.

The function applies the rotation and translation, divides by depth, and projects through the camera matrix - producing nparray_0 (the 2D image points, shape (N, 1, 2)) and nparray_1 (the Jacobian, the derivative of the projection, used by optimizers). The optional imagePoints and jacobian inputs are out-parameters; per the pack README, leave them unconnected.

When you'd actually use it

Classic jobs: after you've estimated a pose with solvePnP, projectPoints tells you where the object's 3D model corners should appear in the image - which is how you draw a 3D bounding box or an "axes gizmo" onto a photo. It's also the forward pass inside calibration and hand-eye calibration pipelines, and it's how you verify a pose estimate is sane. In ComfyUI terms, think of it as the geometry engine behind any "project something 3D onto this frame" workflow.

Install and the honest gotchas

Install once for the whole pack: ComfyUI Manager → search "opencv-comfyui", or

cd ComfyUI/custom_nodes
git clone https://github.com/geroldmeisinger/opencv-comfyui

restart, then pip install opencv-contrib-python.

Three things will bite you. First, every input is an NPARRAY - including cameraMatrix and rvec/tvec, which you must construct as numeric arrays, not type into a widget. This pack has no nice way to build a 3×3 matrix, so you'll likely be generating these in a small Python step or a math node. Second, data types: OpenCV wants float32 arrays, and a wrong dtype produces cryptic assertion errors. Third, the pack only handles batch_size==1 - and honestly, if you're at the point of needing projectPoints_0, you're already used to its dragons.

The verdict

There's a projectPoints_1 twin that's identical (generated from a second overload). Both are solid ports of a rock-solid OpenCV function. They're also a niche so narrow that most ComfyUI users will never open them - which is fine. The pack's author, Gerold Meisinger, said it plainly on release: these nodes are for "small and quick transformations", not full OpenCV apps, and "expect dragons." projectPoints_0 is the honest end of that spectrum: real capability, zero hand-holding, and a steep setup cost you should only pay if you actually do calibration work.