cv2.projectPointsSepJ

This is the variant of cv2.projectPoints that returns the projection and every partial derivative it computes, separately, instead of bundling them into one Jacobian matrix. It is a solver's tool. If you came here because you want to draw 3D points on an image, you want the plain cv2.projectPoints node instead and you can stop reading. If you're building a custom optimiser inside a ComfyUI graph - bundle adjustment, a hand-rolled pose refinement, a hand-eye calibration - this is the overload that gives you the gradients.

What it does

The projection itself is identical to the familiar one: world points, a Rodrigues rotation vector, a translation, the intrinsic matrix K, optionally distortion coefficients, optionally a fixed aspect ratio. What changes is the return: instead of one Jacobian, you get the projected points plus a derivative with respect to each input the model depends on. Those are the pieces a Gauss-Newton step needs, and they're the reason OpenCV exposes this overload at all.

The honest caveat: the pack can't name these outputs. OpenCV's own docs are thin on this one, so the wrapper emits seven generic sockets - nparray_0 through nparray_6 - with blank tooltips. Compare that with the plain projectPoints node next door, which at least reaches imagePoints and jacobian. The first output is your projected points; the rest are the per-input derivatives (rotation, translation, focal length, principal point, distortion), but the order is not something the pack promises. Check the shapes with Inspect CV Data before you wire anything into arithmetic, and don't build a workflow that assumes an ordering you haven't verified on your own install.

Inputs

Same five as the standard projection, and all of them are strict:

• objectPoints - Nx3 (or 3xN) world points, NPARRAY.
• rvec - 3x1 rotation vector, from CV Rodrigues or a solver.
• tvec - 3x1 translation.
• cameraMatrix - 3x3 K, from CV Camera Matrix, CV Load Camera Params (JSON), or a calibration run.
• distCoeffs - required here rather than optional, unlike the plain overload. Distortion vector, or an empty/zero array if the camera is ideal.
• aspectRatio - optional float, default 0, "don't fix the ratio".

Seven NPARRAY outputs. All matrices, no pixels.

When it's worth the trouble

Rarely, and that's the point. The pack ships curated nodes for the work people actually do - CV Project Points (Sequence) for a stack of poses with an optional depth test, CV Unproject Points for the inverse direction, CV Rapid Track for pose tracking through a clip. Those exist because doing this by hand is fiddly and most workflows don't need the gradients.

Reach for projectPointsSepJ when you're computing your own update step: you need the derivative of the projected pixel with respect to the rotation and translation, and doing that numerically in numpy is slower and less accurate than taking the analytic version here. Then hand the result to a solve node, or to the pack's matrix math (CV Matrix Multiply, cv2.SVDecomp) if you're rolling your own least squares.

Install

Part of ComfyUI CV by bmad4ever - roughly 470 auto-generated cv2.* wrappers alongside a few hundred curated nodes. ComfyUI Manager, search comfyui_cv, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Restart afterwards. It needs Python ≥ 3.12 and a recent ComfyUI on the V3 node API, and the OpenCV version is pinned for a reason: the whole pack is curated against opencv-contrib-python-headless 5.0.0.93, and these low-level overloads are exactly the surface where a different OpenCV will behave differently. If you're on an older 4.x wheel, expect this node to be absent or to raise.

Common issues

Blank/missing outputs. All seven derivative sockets are still evaluated even when you only want the points, so a degenerate objectPoints (empty, or the wrong dtype) kills the whole call, not just the derivatives. Validate with Inspect CV Data first.

Wrong-shape matrices. cv2 is unforgiving about (N,1,3) vs (N,3) vs a flat vector; CV Reshape Array is the fix, and it's a normal part of building these graphs rather than a sign you did something wrong.

Node not found. Check your OpenCV version and whether you're on a contrib wheel - tools/repair_opencv_contrib.py --check in the pack directory reports both.