CV ArUco Board Pose (Average)

Anyone who's tried to stabilise a shot or paste a rendered object onto live footage hits the same wall: the model is great at inventing pixels, terrible at knowing where the camera was. ArUco markers solve that geometrically, and this node turns a detected board into the rotation and translation that tells you exactly where a planar target sits in 3D space.

The practical use case in this community is usually video cleanup - paint markers out of footage, then use their geometry to warp a repaired frame across the rest of the clip instead of inpainting every frame. People were hand-rolling that pipeline in other tools years before ComfyUI had anything close; this is the piece that makes it a graph.

How it works

It doesn't just hand the corners to solvePnP and call it a day. Each marker is solved independently with solvePnP using IPPE_SQUARE, which is exact for a coplanar square, then the resulting poses are averaged: the rotation matrices are meaned and re-orthonormalised through an SVD (so the result is a valid rotation, not a slightly skewed matrix), and the translations are meaned to land on the board centroid. Since every marker lies in the same plane, each individual solve already carries the board's orientation, so averaging is a noise-cancelling move rather than a guess.

The two-fold planar-pose ambiguity - a flat target has two valid poses that both project to the same image - is resolved per marker by keeping the solution whose normal faces the camera. That matters: one marker flipped by a noisy detection would otherwise pull the mean somewhere absurd.

Inputs

• corners - from CV ArUco Detect Markers, shape (N, 4, 2).
• camera_matrix - the 3×3 intrinsics from CV Calibrate Camera (ChArUco) (or any of the other calibration nodes, or CV Camera Matrix if you're approximating).
• marker_length - the physical edge of one marker, which defines the world unit. Wire it from CV ArUco Board Image and your translations are in the unit you measured the paper in. Marker length 1 means a cube of size 3 spans three markers.
• dist_coeffs (optional) - leave it unconnected for an ideal pinhole model.

Outputs

rvec (3×1 Rodrigues rotation) and tvec (3×1 translation to the board centroid), plus found. Wire the pair into cv2.projectPoints and then into a drawing node to overlay a 3D object - the pack's ArUco cube workflow is exactly that.

found is the one that saves you grief. Zero markers does not error: you get found=false and a zero pose. Zero rotation and zero translation are a perfectly plausible-looking pose, so branch on the boolean with a control-flow node (Basic Data Handling's IfElse handles it) instead of feeding garbage into your projection.

Install

Manager → ComfyUI CV, or:

cd ComfyUI/custom_nodes && git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Python ≥ 3.12 and a recent ComfyUI (V3 node API). Keep the contrib wheel: cv2.aruco lives in the contrib submodules, so installing plain opencv-python on top silently removes the whole ArUco family from your node list.

Where people get burned

The camera matrix and the board geometry have to describe the same physical setup. An intrinsics matrix from a different resolution, or a marker_length that's the square pitch instead of the marker edge, both produce a pose that looks fine and is wrong by a constant factor - the classic symptom is a projected cube that tracks the board's rotation perfectly and hovers at the wrong distance. Second: don't expect roll. Average-then-orthonormalise gives you a clean orientation, but with a single planar board the normal is all you've constrained; if you need full six-DoF stability, use more markers or a non-planar arrangement.