CV Preview 3D (Calibrated Camera)

ComfyUI core has 3D preview nodes, and they're fine for looking at a model. This one is for a different job: rendering a model through your calibrated camera, so it lands on top of a real photograph instead of floating near it.

Why the core viewer isn't enough

A generic 3D viewport draws with a standard perspective camera - one field of view, a principal point in the middle, square pixels, no lens distortion. A real camera intrinsics matrix has an off-axis principal point, possibly anamorphic pixels (different fx and fy), and a lens with radial and tangential distortion. Those differences sound academic until you composite a virtual object onto a photograph of a chessboard you calibrated against, and the object sits visibly off-target.

This node builds its projection directly from fx/fy/cx/cy, keeps the roll that core's orbit viewers drop, and replicates lens distortion - k1, k2, p1, p2, k3 - with an overscanned render plus a per-pixel Brown-Conrady warp shader. That last part is what you can't express with a 4x4 projection matrix at all. It's a Three.js viewer living inside the node, so you get to orbit and inspect, but the calibrated pose is the starting point and there's a reset button to get back to it.

Wiring it up

model_3d takes a GLB/GLTF from a 3D loader node. GLB is the format that behaves - a .gltf referencing external buffers and textures won't resolve them out of ComfyUI's temp folder. camera_info takes the camera dictionary from CV Camera Pose To 3D View, which is the bridge from an OpenCV calibration to the core 3D loader format. The underscore extras that bridge smuggles through (_intrinsics, _dist_coeffs, _image_size) are what unlock the faithful projection and the distortion pass; a plain camera dict still renders, but as a simple perspective view with distortion off and the canvas letterboxed to the calibration image's aspect.

The whole point of the node, though, is bg_image. Feed it the photo the camera pose was solved from and the render composites over it in exact register. If the virtual object doesn't sit on the photographed board, your calibration or pose is wrong, and you know immediately. That single test - does the model land where the board is - is worth more than any amount of inspecting matrices.

Outputs

Beyond the live preview it emits data, which is the part that makes it composable. render is the server-side software render (the same camera model, minus grid and gizmo) over the background - ready to composite or diff against the photo. silhouette is the model's coverage as a mask, useful for compositing or bbox extraction. depth is object-relative depth normalised 0–1 over the model's own extent - for depth ControlNets or depth-of-field. depth_metric is the rasterizer's actual z-buffer in scene units, and this is the one to remember: it's what CV Project Points (Sequence) wants in its scene_depth, so anchors on the far side of the model get hidden properly. occlusion shows where the model was cut away by a depth map you supplied.

Two optional extras earn their keep: scene_depth (a metric depth map of the photographed scene) makes real objects in front of the virtual one hide it, with scene_depth_bias to nudge the comparison; and object_transform, a JSON string the viewer's gizmo writes as you drag, so a placement survives the run. CV 3D Object Transform writes that same string from readable widgets and emits the matching 4x4 - worth using, since hand-finding a quaternion for a Y-up model is nobody's idea of fun.

Installing

ComfyUI Manager → search "ComfyUI CV", or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
# restart ComfyUI

Python ≥ 3.12, recent V3-API ComfyUI, and opencv-contrib-python-headless~=5.0.0.93. The viewer itself bundles Three.js (MIT) under the pack's web/lib, so there's nothing to fetch. You will need GLB assets - the pack's own examples load MilkTruck.glb, which ships in example_inputs/; run workflows/01_install_example_inputs.json once to copy the sample media into ComfyUI/input and reload the page.

Gotchas

It's GLB or nothing useful. GLTF with external resources quietly fails to load textures from a temp directory.

Distortion off means you're looking at the wrong picture. If _dist_coeffs is missing from the camera dict the render degrades to a pinhole view, and on a wide lens that's a visible mismatch. Check the calibration bridge is feeding the underscore fields.

Orbiting drops roll until you reset. That's in the tooltip, but people still screenshot a tilted orbit and then wonder why the next run's render output doesn't match what's on screen. The render always uses the calibrated pose.