Nodes/camera-comfyUI/PointcloudTrajectoryEnricher
ComfyUI Node

PointcloudTrajectoryEnricher

Filling in what a camera move exposes, in one node

By Alexankharin·Created about a year ago·Updated about a month ago· 36
PointcloudTrajectoryEnricher
  • pointcloud
  • trajectory
  • enriched_pointcloud
  • debug_image
  • debug_depth
camera_type
horizontal_fov90.00
width512
height512
patch_projection
patch_horiz_fov90.00
patch_res512
patch_phi0.00
patch_theta0.00
prompt
num_inference_steps50
guidance_scale7.50
mask_blur5
voxel_size0.07
min_points_per_voxel3
model_nameDepth-Anything-V2-Metric-Indoor-Base-hf

Here's the problem this node solves: you built a point cloud from a single image, and the moment you move the camera more than a few degrees, you hit holes - the parts of the scene your source image never saw. The pack's answer is to outpaint those holes with a generative model, estimate depth on the filled pixels, and lift the whole thing back into 3D so the cloud grows into a walkable scene. PointcloudTrajectoryEnricher wraps that entire loop into a single node that you just point at a cloud and a trajectory.

It's the one-node version of the pack's pointcloud_inpaint.json workflow, and it's a genuinely impressive amount of machinery in a box: for each pose along the trajectory it projects the current cloud, Flux-inpaints the uncovered regions, runs metric depth estimation, backprojects the filled pixels to 3D, transforms them into world space, cleans, and merges the new points into the running cloud. By the end, a cloud that only knew one viewpoint has been taught the rest of the room.

What it does under the hood

Internally it chains the pack's own nodes: ProjectPointCloud to see the view, OutpaintAnyProjection (Flux inpainting) to fill holes, DepthEstimatorNode (Depth-Anything V2 metric) for geometry, ZDepthToRayDepthNode to fix the depth convention, DepthToPointCloud to lift, TransformPointCloud to place, and PointCloudCleaner to keep the additions tidy. The default model_name is Depth-Anything-V2-Metric-Indoor-Base-hf, which you can swap for another Depth-Anything V2 variant.

The inputs that matter

  • pointcloud - your N×7 starting cloud.
  • trajectory - the [K,4,4] camera path to visit; each pose gets one enrich pass.
  • camera_type / horizontal_fov / width / height - how each view is rendered.
  • patch_projection / patch_res / patch_phi / patch_theta - where the outpaint patch is aimed, same language as OutpaintAnyProjection.
  • prompt - what the inpainter should draw in the holes. A good prompt matters; the model fills open space, so describe the scene's content, not "fill hole."
  • num_inference_steps (default 50), guidance_scale (default 7.5), mask_blur - the Flux inpainting knobs.
  • voxel_size / min_points_per_voxel - cleanup settings for the added points.
  • model_name - the depth model.

Three outputs: enriched_pointcloud (the grown cloud), debug_image, and debug_depth - the latter two are the last processed view, which are worth wiring up just so you can see what's happening while it churns.

What it costs

Reality check: this is a slow, heavy node. Every pose on the trajectory runs a Flux inpainting pass plus a depth-estimation pass. That means it needs the inpainting_flux sibling pack (installed automatically by install.py) and a Depth-Anything V2 model download on first use, and each pass is seconds-to-minutes on a decent GPU. The code caps at 15 trajectory poses per run, which is generous enough to be genuinely slow.

Installing it

It's in camera-comfyUI. Manager → Custom Nodes Manager → camera-comfyUI → Install → restart, or:

cd ComfyUI/custom_nodes
git clone https://github.com/Alexankharin/camera-comfyUI.git
cd camera-comfyUI && python install.py

Make sure install.py completed (it clones the Flux inpainting pack); re-run it if OutpaintAnyProjection reports a missing import.

Common issues

  • Hole-filling looks like gray mush - your prompt is too weak or the patch misses the hole; tune patch_phi/patch_theta and describe the scene content.
  • Model download on first run - Depth-Anything V2 downloads automatically; allow network and a few minutes.
  • Slow - no, really slow - expected. Cut the trajectory to fewer poses, drop patch_res, or reduce num_inference_steps.
CategoryCamera/Trajectory

Inputs (18)

NameTypeDefaultDescription
pointcloudTENSOR
trajectoryTENSOR
camera_typeCOMBO3 options: PINHOLE, FISHEYE, EQUIRECTANGULAR
horizontal_fovFLOAT90.00
widthINT512
heightINT512
patch_projectionCOMBO3 options: PINHOLE, FISHEYE, EQUIRECTANGULAR
patch_horiz_fovFLOAT90.00
patch_resINT512
patch_phiFLOAT0.00
patch_thetaFLOAT0.00
promptSTRING
num_inference_stepsINT50
guidance_scaleFLOAT7.50
mask_blurINT5
voxel_sizeFLOAT0.07
min_points_per_voxelINT3
model_nameSTRINGDepth-Anything-V2-Metric-Indoor-Base-hf

Outputs (3)

NameTypeDescription
enriched_pointcloudTENSOR
debug_imageIMAGE
debug_depthTENSOR