SplatTrajectoryEnricher

You've got a splat world that only covers part of a scene - a single-image cloud from SHARP, or a panorama seeded by SphereSplatSeed. The camera hasn't seen the rest of the room, so the world ends at the view boundary. SplatTrajectoryEnricher grows the world: along a trajectory of camera poses it renders the current splats, detects the uncovered (hole) regions, fills them with Flux outpainting, lifts the filled pixels to new splats with Apple's SHARP model, aligns their scale to the rendered depth, and stitches the new content back into the cloud. Visit enough poses and the world walks itself into existence.

This is the heavy artillery node of the pack - the one used in video_to_4d_walkable_world.json to take a polished static splat world and expand it into something you can actually walk through. It's also the most demanding: it chains the pack's renderer, the Flux inpainting pipeline, and SHARP's image-to-splat model in a loop, once per pose. Expect it to be slow, and expect it to need memory.

How it works, briefly

For each pose in the trajectory (capped at max_views, default 10): render the current splats, build a hole mask from coverage, outpaint the holes with Flux (prompt + num_inference_steps default 28, guidance_scale 5), lift the filled view to splats with SHARP (from the checkpoint, default auto-downloaded), scale-align those splats to the render's depth so they sit in the right world position, keep only the splats that actually cover holes, transform to world frame, and smart-stitch via voxel merging (stitch_voxel_size, default 0.01). hole_min_frac (0.02) skips a view if there's too little to fill - a nice self-protection so it doesn't waste a full outpaint cycle on a view that's nearly covered.

The inputs that matter

• splats - the starting GSPLAT world.
• trajectory - the [K,4,4] poses to visit. This is your growth plan: more, further-apart poses = a bigger world, more time.
• camera_projection / horizontal_fov / width / height - how views are rendered.
• prompt - what the outpaint should draw. In practice this describes the scene's content so fills blend in.
• checkpoint - SHARP weights; <download default> grabs the model automatically, or pick a .pt you've put in your input folder.
• max_views, hole_min_frac, stitch_voxel_size - loop control and stitching.
• cache_flux - default true keeps the multi-GB Flux pipeline loaded between views (fast but VRAM-hungry); disable on low-memory GPUs.
• patch_* - outpaint patch geometry.

Three outputs: enriched_splats (the grown world), plus last_render and last_filled (the last view before/after filling) - wire those to a preview so you can watch it work.

What it needs

This node is the reason the pack's install.py exists. It needs the inpainting_flux sibling pack (auto-cloned), the SHARP submodule (submodules/ml-sharpt, bundled or cloned), the Flux model weights (multi-GB download), and gsplat for fast rendering. If the node refuses to load, re-run python install.py in the pack folder and check the console for which piece failed.

Installing it

Manager → Custom Nodes Manager → camera-comfyUI → Install → restart, or:

cd ComfyUI/custom_nodes
git clone https://github.com/Alexankharin/camera-comfyUI.git
cd camera-comfyUI && python install.py

Common issues

• Huge VRAM spike or OOM - that's Flux + SHARP resident simultaneously. Set cache_flux false and/or shrink patch_res.
• Fills that don't match the scene - the outpaint prompt is doing the describing; if the new splats look wrong, the prompt (or the view) is wrong.
• New splats float or clip into geometry - scale-alignment to the reference depth is doing its best, but a bad render fov makes it worse. Double-check horizontal_fov matches your splats' source.
• It's just slow - that's the honest expectation. Fewer poses, smaller patch_res, fewer num_inference_steps.