MIDI-3D Preprocess

Before MIDI-3D Process can generate a scene, it needs the scene chopped into per-object pieces: a cropped RGB view of each object, a mask for each object, and the full scene image for context. That chopping is this node's entire job. It's the unglamorous middle step, and it's also where most "why is my workflow erroring" pain actually lives.

How it works

You give it two things:

• image - the RGB scene, as a normal ComfyUI IMAGE.
• segmentation - a MASK where each object is a different value. That's the key requirement, and it's why this pack ships its own mask-conversion helpers (RGB to Instance Mask, Combine Masks to Instance): a plain black-and-white mask has one object, not several. Each unique non-zero value in the mask is treated as one instance.

The node converts both to PIL, then split_rgb_mask does the real work: for every unique label it cuts out that object onto a white background (instance RGB), builds a binary mask for it, and keeps a copy of the full scene image. That per-instance set is what the diffusion consumes.

One optional input: do_padding (default false). Turn it on when objects touch the image edges - MIDI-3D pads the scene out so border objects get enough room to be generated in full instead of being cut off mid-object. The tooltip says it plainly: "use if objects touch image edges." If your objects are fully inside the frame, leave it off; padding costs resolution.

The output is a single MIDI3D_DATA bundle that wires into MIDI-3D Process (and, optionally, MIDI-3D Texture, which also reads the per-instance crops).

The error that catches everyone

If the mask is all zeros - no distinct non-zero values - the node raises ValueError: No instances found in segmentation mask. This is the #1 failure mode for new users, and it usually isn't the preprocess node's fault. It means your mask pipeline upstream is wrong: you loaded a binary mask straight into segmentation instead of converting it to per-instance labels, or your color-coded mask came through as a single flat color. Check the chain before this node, not after.

A workflow that makes sense

The pack's own example workflow shows the intended graph, and it's worth copying:

Load Image (segmentation) → RGB to Instance Mask → MIDI-3D Preprocess
Load Image (scene) -------------------------------→ (image input)
                                              ↓
                                      MIDI-3D Process

Load the scene, load a color-coded segmentation, convert it to instance labels, preprocess, generate. If you're starting from real photos instead of the pack's example assets, you'll typically build the segmentation with SAM nodes (this node works great off anything that emits per-object masks) and combine them with Combine Masks to Instance before feeding this node.

One warning: the mask and the image need to be the same dimensions and aligned, or your instances will be cut from the wrong places. If your objects come out geometrically wrong but the prompts are fine, look here first.

It's not flashy, but it's the difference between "MIDI-3D works" and "MIDI-3D errors instantly." Get the mask right and the rest of the pack has a chance.