Nodes/ComfyUI_HYWorld2/🌍 WorldMirror V2 3D Reconstruction Advanced
ComfyUI Node

🌍 WorldMirror V2 3D Reconstruction Advanced

Single image and panoramas to point cloud, depth, and Gaussian splats

By AHEKOTΒ·Created 4 months agoΒ·Updated about a month agoΒ· 74
🌍 WorldMirror V2 3D Reconstruction Advanced
  • model
  • images
  • camera_intrinsics
  • camera_poses
  • depth_prior
  • ply_data
  • depth_maps
  • normal_maps
  • camera_poses
  • camera_intrinsics
  • raw_splats
  • filter_mask
  • gs_filter_mask
β—„use_gsplattrueβ–Ί
β—„target_size952β–Ί
β—„offload_schemenoneβ–Ί
β—„head_frame_chunk_size2β–Ί
β—„head_compute_modeallβ–Ί
β—„gs_param_chunk_size1β–Ί
β—„transformer_mlp_chunk_size0β–Ί
β—„confidence_percentile10β–Ί
β—„apply_sky_maskfalseβ–Ί
β—„filter_edgestrueβ–Ί
β—„filter_splatsfalseβ–Ί
β—„edge_normal_threshold1.0β–Ί
β—„edge_depth_threshold0.030β–Ί
β—„apply_confidence_maskfalseβ–Ί
β—„camera_conditioningpose+intrinsicsβ–Ί
β—„missing_camera_strategyoffβ–Ί
β—„splat_camera_sourcecamera_inputsβ–Ί
β—„splat_color_sourceinput_imageβ–Ί
β—„adaptive_target_sizefalseβ–Ί
β—„apply_model_masksfalseβ–Ί
β—„model_mask_threshold0.50β–Ί
β—„voxel_prune_splatstrueβ–Ί
β—„voxel_size0.0020β–Ί
β—„splat_scale_multiplier1.00β–Ί
β—„splat_opacity_floor0.00β–Ί
β—„debug_logfalseβ–Ί
β—„splat_upsample_modenoneβ–Ί
β—„splat_upsample_size1022β–Ί
β—„splat_upsample_depth_sourcegs_depthβ–Ί
β—„splat_upsample_scale0.0030β–Ί
β—„splat_upsample_scale_modedepth_adaptiveβ–Ί
β—„splat_upsample_depth_scale_strength1.00β–Ί
β—„splat_upsample_depth_scale_max3.0β–Ί
β—„splat_upsample_opacity0.90β–Ί
β—„splat_upsample_voxel_prunetrueβ–Ί
β—„splat_upsample_voxel_size0.0015β–Ί
β—„splat_upsample_max_points9000000β–Ί
β—„splat_upsample_cap_far_bias1.75β–Ί
β—„normalize_camera_poses_to_firstfalseβ–Ί

This is the "I want to see every parameter" version of WorldMirror V2. If you just want a point cloud and a splat, the plain VNCCS_WorldMirrorV2_3D node is your friend. This one exists for when you're fighting a specific artifact, feeding it panorama slices, or squeezing a 360Β° scene onto 16GB of VRAM. Same model, same guts - every dial exposed.

What it actually does

WorldMirror V2 is the reconstruction half of Tencent's HY-World 2.0. Feed it one image - or several views cut from an equirectangular panorama by VNCCS_Equirect360ToViews - and it predicts depth maps, normal maps, camera poses, and a Gaussian splat, then returns a PLY point cloud plus a VNCCS_SPLAT you can render or refine. This is not the "walk around inside a generated world" part of HY-World; the author is blunt that Tencent only shipped the 3D reconstruction piece, and that's what this node wraps. The output is a scene you orbit in a splat viewer - exactly the job gaussian splatting is good at.

There's no diffusion loop here. It's a feed-forward model: per-image heads predict depth, points, normals, and Gaussian parameters in one pass, then the node backprojects depth into world space (using your camera priors or the model's predicted cameras) and assembles the splat. The knob-heavy part is that everything runs in chunks - frames per head, tokens per transformer MLP - so you can trade speed for VRAM.

The inputs that matter

You can safely ignore most of the ~30 widgets here. These are the ones you'll actually touch:

  • model - from VNCCS_LoadWorldMirrorV2Model (downloads tencent/HY-World-2.0, roughly 5GB, on first run; pick bf16 for ~2Γ— over fp32, fp8 if you have an Ampere+ card and torchao).
  • images - one or more views. Single flat images work best; the author's own caveat is that WorldMirror still assembles multi-camera panoramas imperfectly, so expect seam conflicts there.
  • target_size - longest side in pixels, default 952. The author has pushed this to ~1400 on a 16GB card (his own dev rig is a 5070 Ti), but only with the VRAM levers below.
  • head_compute_mode - all computes every head; depth+gs skips points/normals to raise the VRAM ceiling; depth_only skips the native Gaussian output entirely. This is your biggest single VRAM lever.
  • offload_scheme - model_cpu_offload moves weights to CPU between uses. Slower, but it's how 16GB survives big target sizes.

Two worth knowing further down: filter_splats defaults to off because applying the filter masks to splats carves black holes into panorama coverage, and apply_sky_mask uses V2's own predicted depth mask - no ONNX model needed.

Outputs and where they go

The node emits eight outputs. The ones you'll wire up:

  • ply_data (PLY_DATA) - the point cloud; feed it to VNCCS_PLYSceneRenderer to render from arbitrary camera angles, or VNCCS_SavePLY to write it to disk.
  • depth_maps and normal_maps (IMAGE) - handy for a quick look or as inputs elsewhere.
  • camera_poses / camera_intrinsics (TENSOR) - the cameras that align the splat; other nodes in the pack expect them paired with the reconstruction.
  • raw_splats (VNCCS_SPLAT) - pipe into VNCCS_SplatRefiner or the HYWorld2 memory/alignment nodes.
  • filter_mask / gs_filter_mask - debugging views of what got filtered.

Install

Same story as every node in this pack. Easiest route:

# via ComfyUI Manager: search "HY-World 2.0" and Install
# or manually:
cd ComfyUI/custom_nodes
git clone https://github.com/AHEKOT/ComfyUI_HYWorld2
cd ComfyUI_HYWorld2
pip install -r requirements.txt
python install.py

The catch: install.py compiles a vendored fork of gsplat (gsplat_maskgaussian) plus recast/navmesh and PyTorch3D from source. It needs a CUDA toolkit roughly matching your torch, MSVC build tools on Windows, ninja, and git. Do not install upstream PyPI gsplat - the project requires exactly one gsplat package and the fork's rasterizer args (distloss, gauss_masks) aren't in the upstream release. If the build fails, python scripts/build_gsplat.py retries it and prints a CUDA smoke test so you know it worked.

Where people get burned

  • This node has no low_vram_mode toggle. That convenience switch lives on the Experimental node. Here you set head_compute_mode, chunk sizes, and offload yourself - at 1400+ on 16GB you basically must.
  • The display name lies. Both this and VNCCS_WorldMirrorV2_3D_Experimental render as "🌍 WorldMirror V2 3D Reconstruction Advanced" in the menu. The Advanced one adds normalize_camera_poses_to_first (official-pipeline camera normalization for prior poses); the Experimental one adds low_vram_mode. When you search for one, make sure you grabbed the right class name.
  • Panorama seams are a known weakness. If you're getting mismatched geometry at view boundaries, try dropping camera_conditioning to intrinsics_only or none - the tooltip notes panorama poses are rotation-only, so prior poses can actively cause the seam conflicts.
CategoryVNCCS/3D/Advanced

Inputs (44)

NameTypeDefaultDescription
modelWORLDMIRROR_MODELβ€”
imagesIMAGEβ€”
use_gsplatBOOLEANtrueGaussian Splatting output. Requires gsplat>=1.5.3.
target_sizeoptINT952252–4096Longest side in pixels. Experimental high values are VRAM-heavy; use low-VRAM modes above 1400.
offload_schemeoptCOMBOnone2 options: none, model_cpu_offload
head_frame_chunk_sizeoptINT21–8Frames processed at once by depth/point/normal/GS heads. Lower values reduce VRAM, especially for FP8 multi-view runs.
head_compute_modeoptCOMBOallall computes every output head. depth+gs skips points/normals to raise the VRAM ceiling. depth_only also skips native GS.
gs_param_chunk_sizeoptINT11–24Frames processed at once by the Gaussian parameter Conv2d head. 1 uses the least VRAM.
transformer_mlp_chunk_sizeoptINT00–262144Token chunk size for transformer MLPs. 0 disables. Lower values reduce VRAM at high target_size but are slower.
confidence_percentileoptFLOAT100–100Discard bottom N% lowest-confidence points.
apply_sky_maskoptBOOLEANfalseRemove sky. V2 uses its own depth_mask prediction β€” no ONNX required.
filter_edgesoptBOOLEANtrueRemove points at depth discontinuities.
filter_splatsoptBOOLEANfalseApply filter masks to Gaussian splats. Off keeps panorama coverage and avoids mask-carved black holes.
edge_normal_thresholdoptFLOAT1.00.1–90β€”
edge_depth_thresholdoptFLOAT0.0300.001–0.5β€”
apply_confidence_maskoptBOOLEANfalseDiscard the lowest-confidence points using confidence_percentile. Official V2 defaults this off.
camera_conditioningoptCOMBOpose+intrinsicsWhich input camera priors to pass into WorldMirror V2. Panorama poses are rotation-only, so testing intrinsics_only/none can reduce seam conflicts.
missing_camera_strategyoptCOMBOoffWhen camera inputs are missing, derive pseudo input cameras from WorldMirror predictions. stabilize_predicted_intrinsics keeps predicted poses but uses sequence-median focal lengths. reproject_stabilized_predicted also rebuilds pts3d/splat means from depth with those stable cameras.
splat_camera_sourceoptCOMBOcamera_inputscamera_inputs backprojects depths from input cameras using ICP scale. predicted uses the model predicted cameras.
splat_color_sourceoptCOMBOinput_imageinput_image colors every Gaussian from the source view RGB, avoiding black SH artifacts. model_sh preserves the model residual SH output.
adaptive_target_sizeoptBOOLEANfalseClamp target_size to the input resolution, matching the official Gradio flow more closely.
apply_model_masksoptBOOLEANfalseFilter outputs using V2 native depth_mask / gs_depth_mask predictions.
model_mask_thresholdoptFLOAT0.500–1Keep pixels whose native model mask is at least this value.
voxel_prune_splatsoptBOOLEANtrueVoxel-merge Gaussian splats after inference. Matches official saving flow and keeps panorama files smaller.
voxel_sizeoptFLOAT0.00200.0001–0.1Voxel size used when voxel_prune_splats is enabled.
splat_scale_multiplieroptFLOAT1.000.25–4Multiply Gaussian scale before saving. Increase slightly if dense visible surfaces have pinholes.
splat_opacity_flooroptFLOAT0.000–1Minimum Gaussian opacity before saving. Raise for debugging holes caused by transparent splats.
debug_logoptBOOLEANfalsePrint camera/depth/splat alignment diagnostics to the ComfyUI console.
camera_intrinsicsoptTENSOROptional: intrinsics from Equirect360ToViews node.
camera_posesoptTENSOROptional: extrinsics from Equirect360ToViews node.
depth_prioroptIMAGEOptional depth prior matching the input views. Enables WorldMirror cond_flags[1].
splat_upsample_modeoptCOMBOnoneExperimental: replace model splats with dense high-res splats from upsampled depth and high-res RGB.
splat_upsample_sizeoptINT1022252–4096High-res splat grid size. Model inference can stay at target_size=518.
splat_upsample_depth_sourceoptCOMBOgs_depthDepth tensor to upsample and backproject into dense high-res splats.
splat_upsample_scaleoptFLOAT0.00300.0001–0.05Constant Gaussian scale for high-res backprojected splats.
splat_upsample_scale_modeoptCOMBOdepth_adaptiveconstant uses one world-space size; depth_adaptive grows with depth; footprint_adaptive follows local 3D spacing; hybrid uses both.
splat_upsample_depth_scale_strengthoptFLOAT1.000–8Depth growth strength, or local footprint multiplier in footprint/hybrid modes.
splat_upsample_depth_scale_maxoptFLOAT3.01–12Maximum depth-adaptive multiplier for splat_upsample_scale.
splat_upsample_opacityoptFLOAT0.900.01–1Constant Gaussian opacity for high-res backprojected splats.
splat_upsample_voxel_pruneoptBOOLEANtrueVoxel-merge high-res backprojected splats before saving.
splat_upsample_voxel_sizeoptFLOAT0.00150.0001–0.1Voxel size for high-res backprojected splat merge.
splat_upsample_max_pointsoptINT90000000–50000000Depth-aware downsample high-res splats to this many points. 0 disables the cap.
splat_upsample_cap_far_biasoptFLOAT1.750–8Preserve proportionally more far-depth splats when applying splat_upsample_max_points.
normalize_camera_poses_to_firstoptBOOLEANfalseOfficial pipeline behavior for prior cameras: convert input poses to frame-0-relative space with inv(first_pose) @ pose before conditioning/alignment.

Outputs (8)

NameTypeDescription
ply_dataPLY_DATAβ€”
depth_mapsIMAGEβ€”
normal_mapsIMAGEβ€”
camera_posesTENSORβ€”
camera_intrinsicsTENSORβ€”
raw_splatsVNCCS_SPLATβ€”
filter_maskIMAGEβ€”
gs_filter_maskIMAGEβ€”