ComfyUI-Recgen

ComfyUI custom nodes for RecGen: turn a single RGB image + depth map + object mask into a 3D mesh and optional Gaussian splat.

Use this extension when you want image-to-3D or RGB-D to 3D inside ComfyUI without writing Python glue code. It targets creators who already have (or can generate) aligned depth and segmentation, and who want .obj / .ply outputs in the ComfyUI output folder.

| | | |---|---| | License | GPL-3.0 | | CI test report | comfy-test results (updates after Actions publish to gh-pages) | | AI / LLM summary | llms.txt · full context |

Features

• RecGen 2D to 3D — one ComfyUI node for full RecGen inference
• RecGen Load Example Inputs — demo RGB, depth, mask, and intrinsics from bundled examples
• macOS Apple Silicon (MPS) — PyTorch spconv fallback and SDPA attention (no flash-attn required)
• ComfyUI-native I/O — IMAGE / MASK tensors, string output path, overlay preview image

Requirements

• Python ≥ 3.10, PyTorch 2.x
• ComfyUI
• Sibling RecGen inference repo (pip install -e recgen)
• ~10 GB disk for first-time HuggingFace weights (TRI-ML/RecGen)
• CUDA recommended; MPS on Apple Silicon supported; CPU works for small/smoke resolutions

Install

ComfyUI (recommended)

cd ComfyUI/custom_nodes
git clone [email protected]:kaili-yang/ComfyUI-Recgen.git

Install RecGen inference and this extension (adjust paths):

pip install -e /path/to/recgen
pip install -e /path/to/ComfyUI/custom_nodes/ComfyUI-Recgen

Restart ComfyUI. Nodes appear under category RecGen.

Development layout

your-workspace/
├── recgen/              # git clone TRI-ML/RecGen
└── ComfyUI/
    └── custom_nodes/
        └── ComfyUI-Recgen/

Smoke test (no ComfyUI UI)

chmod +x ComfyUI-Recgen/scripts/run_demo.sh
./ComfyUI-Recgen/scripts/run_demo.sh
# or: python3 ComfyUI-Recgen/tests/test_run_node.py

Uses 256×256 example data and matching intrinsics. Outputs: outputs/recgen_outputs/.

Quick workflow

1. RecGen Load Example Inputs → RecGen 2D to 3D (demo), or
2. Wire your own RGB, depth, mask, and camera fx / fy / cx / cy into RecGen 2D to 3D

| Node | Description | |------|-------------| | RecGen Load Example Inputs | Demo assets from recgen/examples/ | | RecGen 2D to 3D | Full reconstruction |

Outputs: posed_mesh.obj, overlay.png, optional posed_gaussian.ply under output/recgen_outputs/.

Depth: metric (metres) or uint16 mm PNG. ComfyUI Load Image depth normalized to 0–1 is auto-rescaled when needed.

Documentation

| Doc | Audience | |-----|----------| | FAQ | Common questions | | Node reference | Inputs / outputs | | llms.txt | Short summary for AI crawlers | | llms-full.md | Detailed context for AI assistants | | AGENTS.md | Coding agents |

FAQ (short)

Do I need RecGen installed separately?
Yes — pip install -e the TRI-ML/RecGen repo; this pack is only the ComfyUI adapter.

Does it work on Mac?
Yes (MPS + CPU fallbacks). First run downloads large weights; use the smoke script at 256² for a quick check.

What inputs are required?
RGB + depth + object mask + pinhole intrinsics (or use the example loader).

More: docs/FAQ.md

Develop

cd ComfyUI-Recgen
pip install -e ".[dev]"
pre-commit install
pytest tests/test_comfyui_recgen.py

Related projects

• TRI-ML/RecGen — upstream model and recgen_inference
• ComfyUI — node-based UI
• ComfyUI Manager — optional installer

Citation / attribution

If you use RecGen weights or method, cite and follow the license of TRI-ML/RecGen. This ComfyUI extension is maintained by Kaili Yang under GPL-3.0.