CV Calibrate Camera (Chessboard)

A camera is not a pinhole, and the whole reason solvePnP output can be trusted at all is that somebody measured how wrong it is. This node gives you that measurement: a 3×3 intrinsic matrix (focal lengths and principal point) plus lens distortion, from a batch of chessboard photos.

Why it belongs in a diffusion graph: it's the difference between a plausible 3D overlay and a correct one. Depth and pose tools in this ecosystem - including the multi-view models that now fuse several frames into consistent geometry and camera poses - are all downstream of knowing the intrinsics. If you're doing 3D compositing, an object tracked onto a board, or a calibrated virtual camera in a 3D preview, this is the node that starts the chain.

How it works

Calibration needs 2D/3D correspondences accumulated across several views, which the pack's raw cv2.* wrappers can't express - so this node manages it internally. Feed it an IMAGE BATCH; each frame is grayscaled, its chessboard inner corners are found and refined to sub-pixel accuracy with cornerSubPix, and the matching planar object points are built from the pattern size multiplied by square_size. Then cv2.calibrateCamera solves for intrinsics and distortion across everything it collected. Views where no board was found are skipped rather than failing the run.

Inputs

• images - a batch of views from different angles. Since unusable views get dropped silently, shoot generously: 3 is the floor, 10–20 gives you a decent fit, and variation in angle and distance matters more than volume.
• pattern_cols / pattern_rows - inner corners, not squares. A standard 10×7-square board is 9×6 inner corners, which is exactly why the defaults are 9 and 6. Miscount by one and detection returns nothing on every frame, and you get found=false with no clue why.
• square_size - physical square size in whatever unit you like; it sets the world scale of everything downstream. Leave it at 1.0 if you only need relative geometry.
• The same physical board must be used across all views.

Outputs

camera_matrix (3×3 float64 K), dist_coeffs (k1, k2, p1, p2, k3), rms_error in pixels, views_used, and found.

rms_error is the quality signal and you should look at it every time. Under ~1 pixel is a good fit; a value in the double digits means your board moved, your corner detection latched onto the wrong board, or you have too few varied views. views_used tells you how many of your photos actually contributed - if you fed 20 frames and 4 were used, your board is too small in frame or too blurry.

Failure is soft, which is the right design but worth planning for: fewer than 3 usable views returns found=false with an identity matrix and zero distortion. That matrix is a valid-looking pinhole with no focal length, and everything downstream will run and be wrong. Wire found into an if/else and don't feed a default K into a pose solve.

What you do with it

camera_matrix + dist_coeffs go into cv2.undistort to fix lens distortion, into a pose solve for 3D tracking, and into CV Camera Pose To 3D View when you want a preview 3D node's camera to sit where your real camera sat.

Install

Manager → ComfyUI CV, or:

cd ComfyUI/custom_nodes && git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Python ≥ 3.12, recent ComfyUI on the V3 node API. No model downloads - the "model" here is a printed board.

Where people get burned

Fronto-parallel photos. Ten shots of the same board at the same distance is close to one shot mathematically: focal length and distance are degenerate when the board never tilts. Tilt it, corner it, get close, get far.

A warped board. Print it flat, glue it to something rigid. Paper curls, and a curled board produces a beautiful low RMS with a subtly wrong focal length - the worst kind of error, because the numbers look fine.