[Inference.Core] OneFormer ADE20K Segmentor

Segmentation conditioning answers a different question than edge or depth conditioning does. Canny and Lineart tell the model what shape things are; depth tells it how far away things are; this tells it what category each region belongs to - sky here, building there, person here, grass there - without saying anything about how any of it should look. It's the closest thing to handing the model a paint-by-numbers layout and letting it decide the actual rendering.

OneFormer is a real transformer model doing the heavy lifting here, not a classical filter - it's a Swin-backbone universal segmentation network, and this node specifically runs the version trained on ADE20K, a 150-category scene-parsing dataset skewed toward interiors, architecture, and general scene layout (walls, furniture, windows, floors, sky, vegetation) rather than discrete foreground objects. That bias matters when you're picking between this and its COCO-trained sibling: reach for ADE20K when your source image is a room, a street, or a landscape and you care about the scene's overall structure; reach for COCO when the image is more about specific objects sitting in a scene.

What you set: image is the only required input, and resolution (default 512) is the sole optional parameter - there's no threshold or confidence knob to tune here, unlike some of the other preprocessors in this pack. The output is a single IMAGE: a color-coded map where each flat color corresponds to one ADE20K semantic class. That map goes into a ControlNet Apply node paired with a Segmentation ControlNet checkpoint, loaded separately.

Here's the part worth knowing before you build a workflow around this: segmentation conditioning had a rough time surviving the shift away from SD 1.5 and SDXL. SDXL's xinsir union model still carries full segmentation support, and it remains one of the two places segmentation conditioning genuinely works well today - the other being a community-trained SAM-based model on Z-Image. But none of the official union models shipped since Flux - not Flux.1, not Qwen-Image, not Flux 2, not Z-Image's own official union - include a segmentation condition. If your workflow depends on layout-level control and you're on a post-Flux base, this is one of the cases where dropping back to SDXL's union is genuinely the better tool, not a compromise.

Because OneFormer is a real neural model rather than a classical CV op, it needs pretrained weights, which get downloaded on first use - expect a real pause (and real disk usage) the first time you run this node, unlike the classical preprocessors in this pack that work instantly out of the box.

Installing it: this ships in LykosAI's ComfyUI-Inference-Core-Nodes pack - LykosAI also makes Stability Matrix, the meta-installer a lot of people use to manage local SD UIs, so this pack's install is more deliberately packaged than a typical hobby repo. Search "ComfyUI-Inference-Core-Nodes" in ComfyUI Manager, or manually: cd ComfyUI/custom_nodes && git clone https://github.com/LykosAI/ComfyUI-Inference-Core-Nodes, then run install.py against ComfyUI's Python environment, or pip install -e .[cuda] / .[cuda12] / .[rocm] / .[directml] / .[cpu] matched to your hardware for accelerated onnxruntime, or a plain pip install -e . without it. Restart ComfyUI once done.

Troubleshooting: beyond the pack-wide ModuleNotFoundError: No module named 'inference_core_nodes' issue (a failed or wrong-interpreter Python install step - common on portable ComfyUI builds, target python_embeded there), the OneFormer-specific gotcha is the first-run weight download failing silently on a flaky connection - if this node hangs or errors the very first time you run it, check whether the weights actually finished downloading before assuming it's broken. And if your ControlNet output looks confused rather than obviously wrong, double check you loaded a genuine Segmentation ControlNet checkpoint - a Depth or Canny model fed this kind of flat color-block map won't know what to do with it.