cv2.projectPoints

Here's the whole node in one sentence: give it points in 3D, a camera pose, and a camera's intrinsics, and it tells you which pixel each point lands on. That's the pinhole camera model, and it's the single most useful piece of geometry in this entire pack - every overlay, every AR annotation, every "is my pose actually right?" check runs through it.

What it's for

Three jobs, in order of how often you'll do them:

• Verifying a pose. Project a model's corners through a solved rvec/tvec and draw them. If they land on the object in the photo, your pose is right. If they drift, it isn't.
• Annotation. Project known 3D points - a bounding box, a mesh's vertices, a target marker - into image space, then draw markers or a wireframe on the frame.
• Synthetic overlays. Anything where a 3D thing has to sit convincingly in a 2D image.

This is also the hinge between the pack's two halves. The feature/calibration nodes produce matrices; the drawing nodes consume pixel coordinates. cv2.projectPoints is where matrices become coordinates.

How it works

The inputs are all data arrays, and the node is strict about it - objectPoints, rvec, tvec and cameraMatrix are NPARRAY-only sockets, so you can't accidentally wire a picture into them. Where do those come from?

• objectPoints - an Nx3 (or 3xN) array of world-space points. CV Points, CV Grid Points (a planar calibration grid), CV Mesh From 3D Model's vertices, or the output of a triangulation node.
• rvec - the rotation as a Rodrigues vector. From CV Rodrigues if you have a rotation matrix, or straight out of a solvePnP/calibration node as a 3x1.
• tvec - a 3x1 translation.
• cameraMatrix - the 3x3 intrinsic matrix K. Build one by hand with CV Camera Matrix (fx, fy, cx, cy), read one from a calibration workflow, or load it with CV Load Camera Params (JSON).

Optional distCoeffs is the Brown-Conrady distortion vector - leave it unconnected and OpenCV assumes no distortion, which is the right call for a synthetic camera and the wrong one for a real phone lens. Optional aspectRatio (default 0) only matters if you're asking for the Jacobian with a fixed fx/fy ratio; leave it alone.

Outputs are imagePoints - the projected Nx2 coordinates, which is what you'll draw - and jacobian, the derivative of the projection with respect to the inputs. Nobody needs the Jacobian by hand; it exists for solvers. If you don't know why you'd want it, don't wire it.

A practical note

The projection is only as good as the calibration. A one-pixel error in the principal point, or a K built for 1024×1024 fed points from a 512×512 render, and everything lands slightly wrong in a way that looks like a pose failure. Scale your K and your points to the same image resolution - that mismatch is the number-one cause of "the overlay is nearly right".

Also: this function is a single call, not a batch loop. If you have a whole clip or a stack of poses, use CV Project Points (Sequence), the curated node that projects one point set through an entire batch of rvec/tvec pairs (with an optional depth test against a scene depth map). It's the one you actually want for video; this raw node is for one pose at a time.

Install

From ComfyUI CV (bmad4ever). ComfyUI Manager → search comfyui_cv, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Restart. Python ≥ 3.12, recent V3-API ComfyUI. No models.

Common issues

"Overload resolution failed" or a shape complaint. The arrays have to be the shapes OpenCV expects: Nx3/Nx1x3 points, a 3x1 (or 1x3) rvec, a 3x3 K. Flattened or (N,1,3)-shaped arrays are the usual culprit - CV Reshape Array and CV Array Size are the plumbing nodes for that, and Inspect CV Data tells you what you actually have before you guess.

Output is empty or blank. imagePoints is an Nx2 array of floats, not an image. Nothing renders until you draw it - CV Draw Points for markers, or CV Rapid Track-style wireframe drawing if you're doing meshes.

Everything projects to one spot. Distortion coefficients in the wrong units, or points already in camera space being transformed twice.

Nothing renders in the Python environment at all. If this pack's nodes are missing rather than erroring, suspect the OpenCV wheel: opencv-python installed over opencv-contrib-python silently drops contrib submodules. tools/repair_opencv_contrib.py --check then --apply.