MV-SAM3D Run Multi-View (unofficial)

This is the whole reason the pack exists. The Load Views and Scene From Dir nodes just prepare scenes; MV-SAM3D Run Multi-View is where your several photographs of an object actually fuse into a single mesh (.glb) and/or Gaussian splat (.ply). It wraps the official MV-SAM3D code (arXiv:2603.11633) - the multi-view extension of Meta's SAM 3D Objects - running the authors' InferencePipelinePointMap.run_multi_view unmodified. The "unofficial" in the node name is about this ComfyUI wrapper, not the research: the method underneath is straight from the paper, and it's brand new - nobody's writing tutorials about it yet.

Conceptually it's a two-stage diffusion reconstruction: stage 1 builds sparse structure, stage 2 (a SLAT-style latent pass) refines it, and the fusion weights each view by attention entropy - the paper's headline trick for deciding which camera angle to trust for each part of the object. You can leave the weighting alone; the defaults are the paper's own settings.

How it actually runs

The clever part is under the hood. This node runs in ComfyUI's own Python, but the heavy lifting happens in a subprocess launched with the sam3dobjects-nodes pixi interpreter - the isolated environment ComfyUI-SAM3DObjects already built (pytorch3d, spconv, flash-attn, the works). It writes a JSON job, spawns worker/mvsam3d_worker.py, streams its output, and raises on failure with the worker's last 30 lines instead of a mysterious empty result. The point: keep that multi-GB CUDA stack out of ComfyUI's own process and never rebuild it.

Two things to know before your first run:

• The pipeline is rebuilt from scratch every run - roughly 3 minutes of build, then 4–5 minutes of inference on a 16 GB card. That's normal, not a hang.
• Stage-wise CPU offloading is on by default and is what makes it fit 16 GB (measured 13.95 GB peak instead of ~20 GB fully resident). Disable it only on 24 GB+ cards.

The inputs that matter

• scene - from Load Views or Scene From Dir.
• seed - defaults to 42, the paper's own example seed. Change it to vary results.
• stage1_steps / stage2_steps - denoise steps (50/25 by default). Fewer = faster and rougher; the paper values are a sane starting point.
• stage2_weight_source - entropy is the paper's main method and needs nothing extra. visibility and mixed require a Depth-Anything-3 pointmap .npz fed into da3_npz (an array pointmaps_sam3d of shape (N,3,H,W) plus image_files, saved without pickled objects) - and producing that file needs a separate DA3 install this pack deliberately doesn't automate. Unless you already have DA3 wired up, leave it on entropy.
• decode_formats - comma list, gaussian,mesh by default: gaussian → the .ply splat, mesh → the .glb. This is the input the Export node's errors point back at.
• filename_prefix - output file prefix (letters/digits/._- only), default mvsam3d.
• offload (optional) - on by default; see above.

The three outputs are result (an MVSAM3D_RESULT you pass to MV-SAM3D Export), plus raw glb_path and ply_path strings if you want to skip the Export node and use them directly.

Installing - read this twice

Search ComfyUI Manager for "MV-SAM3D (Unofficial Wrapper)" (repo robinduckett/ComfyUI-MVSAM3D-Unofficial) or:

cd ComfyUI/custom_nodes
git clone --recursive https://github.com/robinduckett/ComfyUI-MVSAM3D-Unofficial.git
cd ComfyUI-MVSAM3D-Unofficial
python scripts/setup_env.py

That one-time setup_env.py installs the extra runtime deps into that env - lightning, roma, a pinned MoGe build, a kaolin stub where no wheel exists, and more.

The real prerequisite, though, is ComfyUI-SAM3DObjects installed and run once so it has produced at least one single-view mesh. That pack provisions the pixi environment (a multi-GB download/build) and downloads the SAM 3D Objects weights, including the pipeline.yaml this node auto-discovers. Skip it and every node here fails with the "could not find the sam3dobjects-nodes pixi environment" error. The pack's own example_workflows/mvsam3d_paper_example.json queues the whole chain against the paper's bundled 8-view stuffed-toy example with zero extra files - that's the fastest way to confirm your setup actually works.

Troubleshooting

• CUDA out of memory - keep offload on, close other GPU apps; the run peaks around 14 GB and cards under 16 GB are untested.
• ModuleNotFoundError inside the worker (lightning, roma, moge…) - the env extras are missing; re-run python scripts/setup_env.py.
• "Could not find … pipeline.yaml" - run ComfyUI-SAM3DObjects once so the weights download, or set MVSAM3D_PIPELINE_YAML.
• Overrides - node inputs pixi_python / repo_root / pipeline_yaml beat env vars (MVSAM3D_PIXI_PYTHON etc.) which beat auto-discovery.
• ComfyUI's interrupt kills the worker cleanly at its next output line.

One licensing note: the wrapper code is MIT, but the MV-SAM3D checkout (pulled as a pinned submodule) and the SAM 3D Objects weights are under Meta's SAM License - broad commercial use, but field-of-use restrictions and share-alike. Read it before shipping.