Interior Design Segmentator

Interior Design Segmentator is the reason the Interior Design for ComfyUI pack exists. Every other node in the pack is scaffolding around it. Feed it a photo of a room and it answers the question that makes virtual staging possible: what is every single pixel? Out the other end come two things the rest of the workflow can't live without - a color-coded segmentation map that steers a ControlNet, and a mask that says exactly which pixels are safe to repaint.

The whole thing is a ComfyUI port of StableDesign, the 2nd-place solution to AICrowd's Generative Interior Design 2024 challenge. The competition was about taking an empty or half-finished room and rendering it furnished and photoreal. The winning idea was to stop treating the room as a flat picture and start treating it as a labeled scene: walls, floor, windows, doors, and furniture each get a class, the diffusion model gets that layout as guidance, and the result is a room that keeps its architecture while everything else gets redesigned. This node is the "label the scene" half of that idea.

How it works

Under the hood it's a semantic segmentation model loaded through Hugging Face transformers: openmmlab/upernet-convnext-small, an UperNet head on a ConvNeXt-Small backbone trained on the ADE20k dataset. It's not a small thing - ADE20k covers 150 classes, from wall and floor to windowpane, sofa, and painting. Each pixel gets labeled, then recolored using the ADE20k palette, producing the recognizable psychedelic segmentation map.

The mechanism for "don't touch the windows" comes from the Control Items node wired into its control_items input. The segmenter looks up the colors of the protected classes, removes them from the map, and builds the mask from what's left. Checked items in Control Items are excluded from the mask; everything else becomes fair game for the redesign.

The inputs and outputs that matter

• image - your IMAGE. One note from the source: it processes images[0] - the first image of a batch. Feed it a single room photo and you're fine; feed it a batch and only the first one gets segmented.
• control_items - the CONTROL_ITEMS output from Control Items. This is mandatory, not optional. No items wired in, and the mask defaults to covering everything except the four default-protected classes (window, door).

Outputs:

• IMAGE - the full-color segmentation map (seg_cond), normalized to 0–1, which is what ComfyUI's ControlNetApply expects. This wires into the segmentation ControlNet (own_controlnet(StableDesign).safetensors), giving the sampler the room layout to respect.
• MASK - a binary 0/1 mask of the redesignable regions. This is what you feed into the inpainting or compositing stage, often after the latent-noise refinement step the workflow uses to smooth edges.

One quirk worth knowing: the MASK comes back on CPU on purpose. The author moved it there to dodge a bug when the mask is used with the IP Adapter node. Don't be alarmed if it isn't sitting on CUDA - that's deliberate.

Install

This is the node that pulls real weight during setup, because it downloads a model on first use:

cd ComfyUI/custom_nodes
git clone https://github.com/RodrigoSKohl/InteriorDesign-for-ComfyUI

Restart ComfyUI (or install "Interior Design for Comfyui" from ComfyUI Manager), then run the node once. On first execution it fetches openmmlab/upernet-convnext-small from Hugging Face - a few hundred MB, and it needs internet. It then loads into VRAM through the pack's transformers dependency. The pack's requirements.txt (torch, transformers, Pillow, numpy, scipy) covers it; scipy is there for the mask-blur step.

The full StableDesign workflow needs more than this pack: Depth Anything V2 and ComfyUI_IPAdapter_plus (the latter is unmaintained - the author says so himself and plans a native replacement), ComfyMath, plus the two custom ControlNet models (controlnet_depth, own_controlnet) and the IP Adapter weights from the MykolaL/StableDesign Hugging Face space. Budget for a big download if you're running the whole thing.

Troubleshooting

• First run hangs. That's the model downloading. Watch the console; it prints "Loading segmentator...". Give it a few minutes on slow connections.
• Segmentation looks wrong - a wall labeled as ceiling, a window missed. That's the ADE20k model, not you. It's a general scene segmenter, not one trained on interiors specifically, so edge cases happen. No tuning parameter fixes it.
• Mask covers more than you want. Revisit your Control Items toggles. Only four classes are protected; everything else is redesignable.
• "Something about IP Adapter bug." If a mask-related error mentions devices, remember the mask is intentionally on CPU - move it explicitly if another node insists on GPU.