SHARP Predict Gaussians (Image + Metric Depth)

Bring your own depth, get consistent gaussians

By PozzettiAndrea·Created 8 months ago·Updated 2 months ago· 358

model
image
metric_depth_layer0
metric_depth_layer1
extrinsics
intrinsics
mask

ply_path
ply_folder
extrinsics
intrinsics

◄snap_to_external_depthnone►

◄depth_conventionplanar_z►

◄focal_length_mm30.0►

◄output_prefixsharp_aligned►

◄save_background_layertrue►

◄edge_crop0.00►

This is the most advanced node in the pack, and it exists to solve a specific problem: SharpPredict guesses its own depth independently every time it runs, which is fine for one photo but a real problem the moment you're generating Gaussians for several overlapping panorama faces - each face's independent guess doesn't agree with its neighbors, and you get visible seams where they should meet. This node fixes that by letting you substitute an external depth map - one you've already reconciled across views with AlignDepthMaps and ProjectDepthToPanorama - in place of SHARP's own monodepth guess. It's the node the README's third example workflow, "Panorama gaussians," actually leans on.

How it works

Three settings do the real work here, and getting any of them wrong produces plausible-but-wrong geometry rather than an error, so it's worth reading them carefully.

depth_convention (default planar_z) tells the node how to interpret your incoming depth. planar_z is depth along the optical axis - native to SHARP and to most planar monodepth models (MoGe2, DA3). ray_distance is Euclidean distance along each pixel's viewing ray - the equirect/panorama convention, what you get after cutting faces out of a merged depth panorama. Pick the wrong one and gaussians distort toward or away from the camera, worst at face edges - the node's own documentation puts the error at roughly 73% at a 90° corner. If your depth came from ProjectDepthToPanorama, route it through SharpRayToPlanarDepth first, or set this to ray_distance directly.

snap_to_external_depth (default none) controls how rigidly the output follows your depth versus the model's own learned predictions. none treats your depth as an input the decoder considers but doesn't have to obey exactly. z_only forces each Gaussian's depth to match yours while preserving the model's learned sub-pixel xy drift (so layer 1 still varies in appearance from layer 0). z_only_firstlayer applies that same snap only to the front surface, leaving the back/occlusion layer at the model's own prediction - useful when you trust your depth for the visible surface but want SHARP's hallucinated occlusion fill left alone. xyz_full and xyz_full_firstlayer are the most rigid: gaussians sit exactly on the pixel-grid ray at your depth, no learned drift at all.

The inputs and outputs that matter

model / image / metric_depth_layer0 (required) - your external depth for the front surface.
snap_to_external_depth (default none), depth_convention (default planar_z) - see above.
metric_depth_layer1 (optional) - if unwired, layer 1 duplicates layer 0 (the tooltip calls this "legacy behavior"); wire it in to restore real two-surface geometry, e.g. from SharpPredictMetricDepth's own layer-1 output.
save_background_layer (default true) - false halves your Gaussian count by dropping layer 1 entirely; useful once you're voxel-deduping downstream, or the scene has little occlusion to begin with.
edge_crop (default 0, range 0–0.5) - trims a fraction off each edge before saving, because SHARP tends to hallucinate poorly at silhouettes and frame edges, and adjacent panorama faces already overlap, so a small crop drops the worst-quality region without losing coverage.
extrinsics / intrinsics / mask / focal_length_mm / output_prefix - same roles as the rest of the pack.

Outputs: ply_path, ply_folder (for batches), extrinsics, intrinsics.

Installing it

Manager: search "Sharp," install highest (nightly if flaky). Or Git URL https://github.com/PozzettiAndrea/ComfyUI-Sharp.git. Manual: clone, pip install -r requirements.txt --upgrade, python install.py, restart.

Common issues

Depth-convention mismatches are the thing to watch, precisely because they don't error - they just quietly warp the result, worst right at panorama seams. If your merged scene looks fine in the middle of each face but distorted at the edges, check depth_convention before anything else. Beyond that, this node inherits the pack's baseline issues: ComfyUI-GeometryPack needed for viewing, and the usual VRAM headroom concerns reported on 8GB cards.

CategorySHARP

Inputs (13)

Name	Type	Default	Description
model	SHARP_MODEL_CONFIG		—
image	IMAGE		—
metric_depth_layer0	IMAGE		External per-face metric depth for LAYER 0 (visible/front surface). Any resolution; auto-resized to SHARP's internal 1536². Replaces SHARP's own monodepth layer 0 — the gaussian decoder's `init_model` sees this as the scaffold for layer-0 base values. When `metric_depth_layer1` is left unwired, this same depth is duplicated into layer 1 (legacy behavior: layer-1 gaussians collapse onto the front surface; only the prediction head's sub-pixel xy / color drift differen-tiates them). Wire layer 1 separately to get real 2-surface geometry.
snap_to_external_depth	COMBO	none	Force gaussian positions to exactly match the external depth in NDC space (applied after gaussian_composer, before unprojection). z_only (recommended): scale each gaussian by target_z/current_z. The NDC xy ray is preserved (so the prediction head's sub-pixel xy drift survives — layer 1 still varies in appearance from layer 0), but depth is forced to match external. z_only_firstlayer: snap ONLY layer 0 to the external depth (same proportional z rescale as z_only); leave layer 1 (and any further layers) at the decoder's predicted z. The front surface is pinned to your depth while the back gaussians stay where the model put them -- preserves SHARP's hallucinated occlusion backplate. xyz_full: reset xy to the pixel-grid identity AND set depth to external. Every gaussian sits exactly on the back-projected ray of its source pixel at the external depth — no learned drift at all. Most rigid; layer 0 and layer 1 collapse to identical positions. xyz_full_firstlayer: apply the xyz_full reset to layer 0 only (front surface sits exactly on the pixel-grid ray at the external depth); leave layer 1+ at the decoder's predicted position (occlusion backplate preserved). none: skip the snap entirely — trust the gaussian decoder's predicted z (the direct-monodepth-override behavior is in effect but not enforced).
depth_convention	COMBO	planar_z	Convention of the incoming `metric_depth`. planar_z (default): depth along the face's optical axis — what SHARP's `unproject_gaussians` and the internal monodepth use natively. Pick this when the depth comes straight from SharpPredictMetricDepth on a single face, or from any rectilinear monodepth model (MoGe2, DA3, etc.). ray_distance: Euclidean distance from the camera center along each pixel's ray — the equirect convention. Pick this when the depth was extracted via SharpDepthMerge -> SharpPanoramaIcosahedronSplit, i.e. when each face was cut out of an equirect depth panorama. The node multiplies by the per-face cos-map `1 / sqrt(((u-cx)/fx)² + ((v-cy)/fy)² + 1)` to convert. At a 90° face corner the difference is ~73% — feeding the wrong convention will distort gaussians toward / away from the camera at face edges.
metric_depth_layer1opt	IMAGE		OPTIONAL external metric depth for LAYER 1 (back/occluded surface). When wired, layer 1 of `monodepth_override` uses THIS instead of duplicating layer-0. Restores SHARP's native 2-surface semantics (layer 0 = visible, layer 1 = behind it) using your external depths instead of the model's predictions. Typical sources: SHARP's own `SharpPredictMetricDepth.layer_1_metric_depth` (model-hallucinated back surface), or `layer0 + small_offset` (e.g. +0.1 m) for a synthetic back-plate, or any other depth predictor. Unwired (default): layer 1 = layer 0 (today's behavior).
focal_length_mmopt	FLOAT	30.00–500	Focal length in mm (35mm equiv). Ignored if intrinsics provided.
output_prefixopt	STRING	sharp_aligned	Prefix for PLY filename or batch folder name.
save_background_layeropt	BOOLEAN	true	SHARP emits 2 gaussian layers per pixel: layer 0 (visible/front surface) and layer 1 (back/occluded). True (default): save both layers — 2× the gaussian count, captures hidden surfaces revealed by parallax. False: save only layer 0 — halves the gaussian count (768² per face instead of 768²×2). Use False when you'll voxel-dedup downstream anyway, or when the scene has minimal occlusion (open outdoor).
extrinsicsopt	EXTRINSICS		Per-face extrinsics from SharpPanoramaIcosahedronSplit. Must match image batch size if batched.
intrinsicsopt	INTRINSICS		Per-face intrinsics from SharpPanoramaIcosahedronSplit (pixel-K for the ORIGINAL face image resolution, not the merged depth resolution). If absent, focal_length_mm is used.
maskopt	MASK		Optional per-face mask (B, H, W). Gaussians at pixels where mask < 0.5 are deleted. Must match image batch size. Use PanoramaSplit's 'create_masks' to generate voronoi-style masks that remove overlap between adjacent faces.
edge_cropopt	FLOAT	0.000–0.5	Fraction of each edge to drop before saving the PLY (per side). 0.0 = no crop (default), 0.1 = drop the outer 10%% of width AND height (keep the central 80%% × 80%% gaussians). Useful to discard edge gaussians that SHARP often hallucinates at silhouettes / frame edges, especially when stitching panorama faces — adjacent faces overlap, so cropping a few percent off each removes the worst-quality region without losing scene coverage. Applied per-layer to the H×W pixel grid so both layer 0 and layer 1 keep the same central region.

Outputs (4)

Name	Type	Description
ply_path	STRING	—
ply_folder	STRING	—
extrinsics	EXTRINSICS	—
intrinsics	INTRINSICS	—