Nodes/comfyui_cv/CV Unproject Points
ComfyUI Node

CV Unproject Points

Turns pixel positions plus a known depth into 3D camera-space points - the inverse of 'cv2 projectPoints' when you already know how far away each point is. 'CV Depth to 3D Points' does this for a WHOLE depth image; this does it for the handful of points you actually care about (a click, a detection, a contour vertex), which is what turns a click on a rendered object into a 3D anchor on it. Depth is camera-space Z (distance along the optical axis), not distance from the camera centre - the same convention as 'CV Rasterize Mesh'. Points whose depth is non-finite or <= 0 come back as NaN rather than raising, so a click that missed the object stays a valid row you can filter out.

By bmad4ever·Created 3 months ago·Updated 2 days ago· 0
CV Unproject Points
  • points_2d
  • depths
  • K
  • dist_coeffs
  • points_3d
  • valid
  • count
Categoryimage/CV/low-level

Inputs (4)

NameTypeDefaultDescription
points_2dNPARRAYNx2 (or Nx1x2) pixel positions. 'CV Annotate Points' produces these from clicks on the image.
depthsNPARRAYN camera-space Z values, one per point - sample them out of a depth map with 'CV Sample Array At Points'. A single value broadcasts to every point, which is how you unproject onto a plane at a known distance.
KNPARRAY3x3 camera matrix the points were seen through.
dist_coeffsoptNPARRAYOptional lens distortion of that camera. When wired, the points are undistorted first (cv2.undistortPoints), so pixels measured on a real photo unproject onto the ideal ray. Leave unwired for a pinhole render.

Outputs (3)

NameTypeDescription
points_3dNPARRAYNx3 float32 points in CAMERA space (X right, Y down, Z into the scene). To get them into the model's own frame, invert the pose that rendered them - 'OpenCV Pose To Matrix' -> 'cv2 invert' -> 'CV Transform Points 3D' - which is what the 'CV Pick Point On Mesh' blueprint wires up.
validNPARRAYN uint8: 0 where the depth was missing (non-finite or <= 0) and the point came back NaN. A click that landed on the background reads 0 here.
countINTHow many points unprojected successfully.