(Down)Load MIDI-3D Model

Let's be honest about what you're getting into. This is the loader for MIDI-3D, a research model from VAST-AI that turns a single image into a 3D scene containing multiple separate objects - a chair next to a table next to a lamp, each its own mesh, all generated at once. It made the rounds on r/StableDiffusion in June 2025 as a paper drop with a working HF demo, and the general reaction was "okay, that's cool, when can I run it locally." This pack is one answer to that. The README opens with the words "Work in Progress! This node is not finished." Keep that in mind.

The node's name is the whole story: (Down)Load MIDI-3D Model. It's the only loader in the pack, it downloads the weights for you, and everything else in the workflow hangs off its single model output.

What it does

The trick that separates MIDI-3D from the older "image to one 3D object" models (TripoSR, InstantMesh, the various zero123 clones) is multi-instance diffusion. Instead of generating each object in the scene one at a time and hoping they align, the model generates all instances simultaneously, with a custom attention mechanism that lets them talk to each other so they end up with correct spatial relationships. It extends a pre-trained image-to-3D model rather than training a scene generator from scratch.

This node loads that pipeline:

• model - one choice, MIDI-3D. On first run it calls snapshot_download from HuggingFace (VAST-AI/MIDI-3D) into ComfyUI/models/midi3d/MIDI-3D/. That's a real download, multi-gigabyte, so the first Queue hit after install looks like the UI has frozen. It hasn't.
• dtype - bfloat16 (default), float16, or float32. The author's tooltip says bfloat16 for RTX 30xx and newer, which is the right instinct: it's the memory-cheapest option that doesn't fight with Ampere-era tensor cores. Drop to float32 only if you're chasing stability and have VRAM to burn.
• gpu_cache - default false. With it off, the pipeline offloads to CPU and frees VRAM after each generation, which matters because MIDI-3D is a heavy diffuser and your card is also holding the rest of the graph. Flip it on if you're chaining multiple process calls and don't mind the VRAM staying occupied.

Output is a single MIDI3D_MODEL that wires straight into MIDI-3D Process.

Installing it

ComfyUI Manager is the easy path - search MIDI-3D and install the "ComfyUI-MIDI3D" entry. Or clone it yourself:

cd ComfyUI/custom_nodes
git clone https://github.com/PozzettiAndrea/ComfyUI-MIDI3D
cd ComfyUI-MIDI3D
pip install -r requirements.txt
python install.py   # torch-cluster, nvdiffrast, mv-adapter

That last line matters. The README's requirements.txt is the partial list - it omits the native pieces (torch-cluster for the geometry pipeline, nvdiffrast, and mvadapter for texturing) that install.py fetches from PyG's wheel index and GitHub. Skip install.py and you'll hit import errors the first time a node tries to touch 3D geometry. Then restart ComfyUI and let the loader do its thing on first run.

Where people get burned

• It needs CUDA. The node prints a warning if you run on CPU, and "very slow" is doing a lot of work in that sentence. Don't.
• The model download happens at load time, not install time. If the download fails partway, delete models/midi3d/MIDI-3D and queue again - the node re-checks for a valid model_index.json each load.
• The pack is genuinely WIP. It's a wrapper by Andrea Pozzetti around a research model that expects a research-GPU environment. Expect rough edges and check the GitHub Discussions before you file a bug.

One honest take: MIDI-3D is a research artifact, not a production asset pipeline. You'll get hobbyist-grade geometry - good for blocking out a scene, not for a product shot. The loader is the most polished part of the pack, and it's just the door. The real question is whether the 3D output is good enough for what you're building, which is a MIDI-3D Process conversation, not a loader one.