RecGen 2D to 3D

Most image-to-3D nodes in ComfyUI - TRELLIS, TripoSR, the whole "one photo in, mesh out" crowd - hallucinate the back of the object from a single flat RGB frame. RecGen 2D to 3D goes a different, older-school route: it wants a photo plus an aligned depth map and an object mask, then reconstructs actual geometry from that RGB-D data. The depth isn't guessed, it's an input. That's the whole pitch, and it makes this node a different animal from the one-click tools.

This is the workhorse of the kaili-yang/ComfyUI-Recgen pack, a thin ComfyUI adapter around RecGen, Toyota Research Institute's single-view reconstruction model. Feed it image + depth + mask and you get a textured .obj mesh in your ComfyUI output folder - plus, optionally, a 3D Gaussian splat as a .ply.

How it works

The node wraps RecGen's inference with a modest amount of glue. Your image, depth, and mask tensors get converted to numpy, the depth tensor is run through a unit-normalization helper (more on that in a sec), a pinhole camera matrix is built from fx / fy / cx / cy, and the whole thing is handed to RecGen's generate(). Under the hood that's sparse-convolution layers and a DINOv2 image encoder, which is why the pack ships a pure-PyTorch spconv mock and SDPA attention for Macs and CPU boxes.

The model is cached per checkpoint and device, so the first run is brutal - RecGen + DINOv2 weights are ~10 GB - and every run after is faster. Outputs land in output/recgen_outputs/recgen_{timestamp}_{seed}/ with posed_mesh.obj, mesh.obj, overlay.png, and metadata.json. posed_mesh.obj is in the camera frame, mesh.obj is object-centric - the latter is usually the one you want to drag into Blender.

The inputs that matter

The node is genuinely simple once you stop tripping on the intrinsics:

• image / depth / mask - the actual data. Depth and RGB must be pixel-aligned, and the mask should tightly cover your object.
• fx / fy / cx / cy - pinhole camera intrinsics, in pixels. These are the part beginners get burned on: the defaults (1062.2 / 1060.9) match RecGen's own example data at 256², so they're wrong for a 1024×1024 input. The one mercy: cx and cy default to -1, which the tooltip explains means "auto - center of image," and that's usually fine.
• device - cuda, mps, or cpu. The node auto-selects the best available as the default, so you rarely touch it.
• checkpoint - currently just recgen_base.multiview_stereo (it enumerates more once RecGen is importable, but today it's effectively fixed).
• seed - changes the randomized decoding. Fiddle with it if a run comes out lumpy.
• save_splat / save_glb - toggle the Gaussian splat (.ply, on by default) and a textured GLB (off; it drags in extra dependencies).

The outputs

Two: output_dir (a STRING - where everything landed) and overlay_image (an IMAGE preview of the mesh render). Wire overlay_image into a Save Image node to eyeball the result in the UI; the path is how you find the .obj and .ply on disk.

How to install it

This pack has a sibling dependency, and that's the whole install story: it's only a ComfyUI adapter, so you need the upstream TRI-ML/RecGen repo importable too. The pack's install.py handles that by cloning RecGen into vendor/recgen and pip-installing it, but the README's canonical route is:

cd ComfyUI/custom_nodes
git clone https://github.com/kaili-yang/ComfyUI-Recgen
git clone https://github.com/TRI-ML/RecGen
pip install -e /path/to/recgen
pip install -e /path/to/ComfyUI/custom_nodes/ComfyUI-Recgen

Then restart ComfyUI - the nodes appear under RecGen. Via ComfyUI Manager, search "ComfyUI-Recgen" (publisher kaili-yang); Manager will run install.py, which clones RecGen. Either way, budget ~10 GB of disk for the first-run HuggingFace downloads.

Common issues

• Depth looks wrong in the mesh. Depth and RGB must be pixel-aligned; the mask should hug the object; and the intrinsics have to match the resolution of the tensors you feed, not the defaults.
• Depth units. The node accepts meters or uint16-mm PNGs. This is where people get confused: ComfyUI's Load Image normalizes everything to 0–1, so the pack auto-rescales a 0–1 depth image by ×65535 back to millimeters. Feed it raw metric meters and it'll assume millimeters.
• First run is slow / huge. That's the ~10 GB of weights downloading once. On CPU it stays slow; use the 256² smoke test to confirm the plumbing first.

RecGen won't replace TRELLIS for "single photo, zero setup" magic - it's not trying to. If you've already got a depth map from Depth Anything or Marigold and want geometry that respects it, this is the node that treats that depth as ground truth.