MIDI-3D Process

MIDI-3D Process is the engine of the pack. The loader hands you a model, the preprocess node hands you cleaned-up per-instance images, and this is where the diffusion actually runs - the node that takes your flat picture of a desk scene and returns a MIDI3D_SCENE with a separate mesh for the monitor, the keyboard, and the mug.

It's the slow one. There's no way around that: multi-instance diffusion generates every object simultaneously through a shared attention mechanism (that's the "multi-instance" part of MIDI-3D - objects talk to each other during generation so they land in coherent spatial relationships), then decodes geometry and runs marching cubes on the result. Expect to wait minutes, not seconds, and to feel every one of the 50 default steps on a mid-range card.

The inputs that matter

• model - the MIDI3D_MODEL from (Down)Load MIDI-3D Model. Nothing else plugs in here.
• preprocessed - the MIDI3D_DATA from either preprocess node. This carries the per-instance RGB crops and masks the model needs.
• num_inference_steps - default 50, range 10–100. More steps, cleaner geometry, slower. If your meshes come out lumpy, this is the first knob to turn, not guidance.
• guidance_scale - default 7, range 1–15. Classifier-free guidance, same concept as your image samplers. Push it up when objects drift from their reference crops.
• seed - default 42, -1 for random. Same seed + same inputs = same scene, which is handy when you're hunting for one that doesn't melt.

The one setting beginners actually need to touch is seed - set it to -1 when you're exploring, then lock a good one in.

What comes out

The output is a MIDI3D_SCENE, but don't be fooled by the custom type name - underneath it's a trimesh.Scene, one mesh per instance. Internally the node runs the diffusion, then extracts each object's geometry from its SDF grid via skimage.measure.marching_cubes, scales the vertices into world coordinates, and packages everything into the scene. It also stashes metadata on the scene: instance count, inference time, steps, guidance, seed. If you save the result and wonder "what settings made this mess," it's right there.

From here the scene feeds two places:

• MIDI-3D Texture - add MV-Adapter textures to the meshes.
• Scene to Trimesh - merge everything into one mesh and export.

The gotcha: VRAM and the offload

Right after generation the node offloads the pipeline to CPU and empties the CUDA cache - unless you set gpu_cache to true on the loader. That means the first run after a fresh load carries a big memory transfer, and if you're running a texture pass right after, the model gets shuffled back onto the GPU. It's the author's chosen trade: free VRAM for the rest of your graph by default, slower when you chain generations. If you're looping seeds back-to-back, flip gpu_cache on and eat the VRAM.

When it goes wrong

• "No instances" upstream - if preprocess found zero objects in your mask, you'll get an error before this node ever runs. Check the segmentation, not this node.
• Blobby, broken geometry - bump steps, check your mask quality, and remember this is a research model. MIDI-3D's own page doesn't promise clean topology; you get mesh soup that looks fine in a viewer and falls apart under close inspection.
• OOM - a heavy diffusion plus marching cubes on a 16GB card is genuinely tight. Lower steps first, then consider a smaller scene (fewer instances means smaller attention cost).

Reasonable expectations: MIDI-3D is the coolest thing in this pack and still a research demo. Use it to block out a compositional scene you'd otherwise fake with individual image-to-3D models - that's the workflow it actually wins at.