FisheyeToGaussian

ImageToSplat is the single-image → Gaussian splat node, and it's great - right up until your image is a 180° fisheye, at which point SHARP's rectilinear assumptions make the whole thing collapse at the edges. FisheyeToGaussian is the wide-angle version: it chops your fisheye view into several pinhole slices, runs SHARP's single-image splat prediction on each, then rotates them into place and stitches them into one splat cloud that covers the full field of view.

For the VR crowd this is the node that makes "one 180° shot → a splat scene you can orbit in a headset" a real thing. It's the geometry backbone of the static-camera fisheye 4D workflow, and it's the reason you'd feed frame 0 of a fisheye video into this rather than ImageToSplat and watch the borders tear.

What you set

• image - your fisheye frame. fisheye_horizontal_fov (default 180°) tells it how wide the lens is.
• output_width / output_height (default 0 = auto) - output splat cloud size; 0 lets the node infer from input.
• checkpoint - a .pt checkpoint from your input folder, or <download default> which fetches the SHARP model for you.
• pinhole_horizontal_fov (default 90°) - the FOV of the individual slices SHARP sees. 90° is the sweet spot for quality vs. count.
• stitch_mode (default smart) - how overlapping slice splats merge: keep / discard / average / smart / main_direction.
• stitch_voxel_size (default 0.01) - voxel resolution for the stitch merge; 0 disables voxel merging.
• stitch_direction_deg (default 5°) - how close in angle two splats need to be to count as overlap.
• feathering (default 0) - softens seam splats if the stitch shows boundaries.

Output is a single splats (GSPLAT) object - the same type every GSplat node in the pack speaks, ready for FuseSplats, RotateSplats, or direct rendering.

The real requirements

This is the pack's SHARP node family, which means the submodules/ml-sharpt checkout and gsplat must be present. install.py sets both up for you (Manager does it automatically), and gsplat JIT-compiles its CUDA kernels on first use - expect a compile pause the first time, and expect it to need a real NVIDIA GPU with matching PyTorch. The checkpoint downloads on first run if you accept the default.

Install

Shared camera-comfyUI install: Manager → search "camera-comfyUI" → Install, or clone https://github.com/Alexankharin/camera-comfyUI into custom_nodes/ and run python install.py. If FisheyeToGaussian shows up but errors at run time, re-run python install.py - a failed SHARP/gsplat step earlier is the usual culprit, and the fix is the same command.

Troubleshooting

• Seams visible where slices meet: raise feathering, switch stitch_mode to smart, and make sure stitch_voxel_size isn't so large it's merging genuinely different geometry.
• Edges of the 180° frame come out empty: the fisheye may not actually cover 180° (some lenses claim it and don't). Lower fisheye_horizontal_fov to match the real coverage.
• First run hangs: gsplat kernel JIT compile, then SHARP weights. Both are one-time; watch the console.
• Whole cloud drifts in VR: check the depth/splat scale convention - pair with a depth-consistent pipeline (FisheyeDepthEstimator + ray-depth conversion) so geometry scale matches what your headset assumes.