CV ICP Register (Point Clouds)

Iterative closest point is the classic way to snap two 3D point clouds together: repeatedly find each point's nearest neighbour in the other cloud, solve for the rigid transform that minimises those distances, apply it, repeat. cv2.ppf_match_3d.ICP is the implementation, and CV ICP Register exposes it for two clouds, returning the 4×4 pose that maps model onto scene - ready to feed into CV Transform Points 3D.

The headline, and the reason this node has a warning attached: ICP only polishes. It is a local optimiser with no notion of the right answer. Give it a good starting alignment and it refines it beautifully. Give it a bad one and it converges, confidently, to nonsense. In practice that means you align first with cv2.estimateAffine3D (or by whatever means you have), and use ICP to take the last bit of error out.

Inputs

model is the N×3 (or N×6 with normals) cloud to be moved; scene is the M×3 (or M×6) reference to align onto. iterations (default 100) is the max per pyramid level. tolerance (default 0.05) is the relative-improvement threshold that stops iteration early. rejection_scale (default 2.5) is outlier rejection - correspondences farther than that many standard deviations are dropped each iteration, and lowering it makes ICP stricter.

num_levels (default 6) deserves its own paragraph, because the tooltip is unusually specific and it's the parameter that causes the most confusing failures. More coarse-to-fine pyramid levels means ICP tolerates a rougher initial alignment - but only up to the point where the coarsest level has nothing left to fit, at which point ICP diverges rather than degrading gracefully. The README's own measurements on partial scenes: a 360-point model against a 176-point single-camera view converges at 4 levels and diverges at 5; a 1363-point model against a 1335-point view converges at 6 and diverges at 8. A depth camera only ever sees one face of an object, so if you're registering a full model against a partial view, you're squarely in the regime where this matters. If found comes back false with a ~1e10 residual, lower num_levels - that's the documented fix.

Outputs, and the one you must not trust

pose is the 4×4 rigid transform (identity when found is false). found is false when ICP couldn't run at all - empty cloud, cv2 error, non-finite pose - or when it diverged. Your transform should be gated on that with a Basic data handling: IfElse, not applied blindly.

And residual? It's not a quality measure, and the node says so in bold, with evidence: on the shipped stereo example it reported 5e-5 while producing a pose worse than its RANSAC initialisation - nearest-neighbour median distance went from 0.34 m to 0.69 m, plus 7.6 degrees of spurious rotation. A residual that looks superb while the alignment degrades is the single most expensive trap in this node. Judge the alignment with CV Point Cloud Nearest Distance and require the median to go down, rather than reading ICP's own number.

The one thing residual does tell you: 9999999999 is cv2's divergence sentinel, and the node converts that into found=false - which matters because registerModelToScene still returns 0 (success) in that case, so the raw return code would have reported success on a pose hundreds of units out.

Two practical habits: downsample first - ICP cost grows with cloud size, so CV Downsample Point Cloud goes in front - and make sure normals are right, since the PPF family wants N×6. If your cloud came from a mesh, take normals from the triangle winding rather than a plane fit, because a plane fit can't orient a closed model and will flip the pose roughly 180°.

Install

ComfyUI Manager, search the pack title comfyui_cv (bmad4ever/comfyui_cv). Or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv

Restart. Requires Python ≥ 3.12 and a recent ComfyUI on the V3 node API - this pack uses io.ComfyNode/io.Schema throughout and defines no NODE_CLASS_MAPPINGS, so an older build loads nothing at all. Dependency:

pip install "opencv-contrib-python-headless~=5.0.0.93"

This node lives in cv2's ppf_match_3d, i.e. the contrib side, which is exactly why the wheel must be contrib. All four OpenCV distributions share one site-packages/cv2, so a non-contrib install over a contrib one silently empties the contrib submodules and this node - along with the rest of the contrib category - disappears from the menu without an error. tools/repair_opencv_contrib.py --check / --apply repairs it.

Bottom line

Reach for ICP when you have a decent initial alignment from elsewhere and want to tighten it: depth-camera alignment, scan stitching, pose refinement from a coarse estimate. Don't expect it to solve registration for you, and never ship a workflow that trusts residual. As ever with this pack, the README's own disclaimers apply - heavy LLM assistance in development, sample-tuned example pipelines, no support, and an explicit warning against production use without review. That warning is doing real work on this particular node.