CV Pose To Matrix

A four-line node that prevents a very specific kind of afternoon. OpenCV hands you poses as a rotation vector and a translation vector; nearly everything else - including the point wrappers and the 3D matching nodes in this same pack - wants a matrix.

What it's for

CV Solve PnP and CV ArUco Board Pose both give you an rvec (a Rodrigues rotation vector) and a tvec (translation). That is the camera's pose in the object's frame, or the object's in the camera's, depending which way you set it up - and it is the format you cannot do anything with directly. This node turns it into the shape you can use.

How it works

cv2.Rodrigues converts the 3x1 rotation vector into a 3x3 rotation matrix, and that gets concatenated with the translation to give [R|t] - a concatenation cv2 has no single function for, which is why the node exists. A 3x3 rotation matrix passed in as rvec is passed straight through, so you can feed it whichever rotation representation you happen to be holding.

Three outputs, and which you take depends on the consumer. matrix_3x4 is the classic [R|t], the thing CV Transform Points 3D takes and uses to map a point between the two frames - forward maps an object-frame point into the camera frame, inverse goes back the other way, which is how you get a stereo point cloud from camera space into object space. matrix_4x4 is the homogeneous version that CV ICP Register and most 3D tooling expect. rotation is just the 3x3 R on its own, for when you only care about orientation - a turntable measurement, a normal check.

Failure is handled the honest way: a missing or non-finite pose gives the identity transform rather than an error, so points pass through unchanged. That's a kindness and a trap in equal measure - a wrong-but-identity result is silent, so if a downstream transform looks like a no-op, check whether you actually got a pose.

Inputs and outputs

rvec and tvec are the only inputs, and both come straight off a pose solver. Outputs are matrix_3x4, matrix_4x4 and rotation, all as NPARRAY.

The workflow this lives in is the augment-reality one: detect an ArUco board or solve PnP on known points, convert with this node, transform a model's vertices or a cube's corners with the result, then project them with cv2.projectPoints to draw the overlay. If you're doing matching rather than rendering, the 4x4 goes to ICP and the 3x4 goes to point transforms.

Installing

Pack-wide install. ComfyUI Manager → search "ComfyUI CV", or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
# restart ComfyUI

Python ≥ 3.12 and a recent ComfyUI on the V3 node API. Dependencies are opencv-contrib-python-headless~=5.0.0.93, numpy, torch - install the contrib wheel, since a plain opencv-python dropped on top of it silently empties the contrib submodules and takes a slice of the pack's nodes with it. python tools/repair_opencv_contrib.py --check diagnoses that.

Gotchas

Which direction does the transform run? The pose from CV Solve PnP maps object points into the camera frame. If your points are already in camera coordinates - a stereo cloud, for instance - you want the inverse direction, which CV Transform Points 3D gives you from the same matrix. Getting this backwards produces a result that's mirrored or rotated about the object and looks almost plausible.

Rotations compose in order. If you're chaining a PPF or ICP pose with a PnP pose, multiply the matrices in the right order and in homogeneous form. CV Matrix Multiply exists for exactly that.

The identity fallback. Non-finite input becomes identity, silently. It's a good default for a live pipeline and a bad one for debugging, so when a 3D overlay has drifted, test the pose upstream of this node before you blame the matrices.