Align Depth Maps (Global Optimization)

Monocular depth has a well-known blind spot: predict depth independently for two photos of the same overlapping scene, and each prediction gets its own arbitrary scale. Nothing forces them to agree, because each view was estimated in a vacuum. That's fine for a single image - but the moment you're cutting a panorama into crops (SamplePanorama, SharpPanoramaCubeSplit) and predicting depth for each one separately, those per-view depths need to be reconciled before you can stitch them back into one coherent scene. That reconciliation is this node's entire job.

How it works

method picks the solver:

• global_optimization (default, recommended) - solves the scale factor for every view jointly, in one pass. If view 3 and view 7 both overlap view 5, the solver accounts for both relationships at once rather than working through them one at a time.
• bfs_chain - chains pairwise alignments together, view by view. The node's own tooltip flags the risk directly: this "can drift," because scale errors compound the further a view is from wherever the chain started. It exists as a fallback, not the default, for exactly that reason.

reference_view (default index 0) is the one view treated as ground truth - everything else gets solved to match its scale (scale = 1.0). Worth choosing deliberately rather than leaving it at whatever index 0 happens to be: pick a well-exposed, geometrically normal view, not one pointed mostly at sky or flat ground, where depth tends to be the least reliable to begin with.

The inputs and outputs that matter

• depth_maps / extrinsics / intrinsics (all required) - the per-view depth maps and their camera matrices, typically straight out of SharpPredictDepth or SharpPredictMetricDepth run across a batch of panorama crops.
• method (default global_optimization).
• reference_view (default 0, range 0–999).

Outputs: aligned_depth_maps (the reconciled result - this is what feeds ProjectDepthToPanorama next), extrinsics / intrinsics (pass-through), and alignment_info (STRING) - a readable summary of what correction each view got, useful for spotting a bad alignment before it propagates further downstream.

Installing it

Part of ComfyUI-Sharp: Manager search "Sharp" (install highest, nightly if needed), Manager → Git URL https://github.com/PozzettiAndrea/ComfyUI-Sharp.git, or manual clone + pip install -r requirements.txt --upgrade + python install.py, restart.

Common issues

alignment_info is your actual debugging tool here - read it before assuming a bad-looking merge downstream is ProjectDepthToPanorama's fault. If one or two views got wildly different scale corrections than the rest, that's usually either a bad reference_view choice or a view whose depth prediction was already unreliable going in (heavy motion blur, a mostly-featureless region, extreme close-up).

If you're getting visible drift or seams specifically with bfs_chain, that's the documented behavior, not a bug - switch to global_optimization unless you have a specific reason to chain pairwise. And since this node only reconciles scale, not convention: make sure the depth you're feeding it is already in the right units for your pipeline (see SharpRayToPlanarDepth if you're mixing ray-distance and planar-z depth from different sources) - alignment can't fix a unit mismatch, only a scale mismatch.