OneFormer ADE20K Segmentor

When you want a segmentation map and you want it right, this is the one to reach for. OneFormer labels every pixel by category - sky, building, person, road - same idea as UniFormer, but with a bigger, more capable model behind it. On busy, cluttered scenes it separates objects more cleanly and mislabels less. If you're using segmentation to lock down the layout of a complex scene, the extra accuracy here translates directly into the ControlNet respecting the regions you actually meant.

How it works

OneFormer is a universal image segmentation model, and this node runs the variant trained on ADE20K - the standard 150-category label set that ControlNet's seg models were built around. It classifies each pixel, paints it the canonical color for its class, and hands you that flat, blocky map. You feed it into a seg ControlNet (control_v11p_sd15_seg on SD 1.5), and the model generates a fresh image where each colored region becomes the thing it's labeled as - while your prompt decides how all of it looks. Same layout, wildly different renders.

The important context: segmentation is one of the conditions that never made the jump to the modern architectures. The Flux, Qwen-Image, and Z-Image unions cover canny, depth, pose, and edges - not segmentation, not normals. SDXL's xinsir union is the newest place seg conditioning still exists. So this is fundamentally an SD 1.5 / SDXL tool in 2026, and that's fine - just know it before you wire it to a Flux generator and wonder why there's no ControlNet for it.

The inputs that matter

Nice and minimal. It takes an image and returns an IMAGE - the segmentation map - for your ControlNet Apply node. The only setting is:

• resolution (default 512) - working size; match it to your render.

There's nothing to tune on the segmentation itself. You pick the model (by picking this node over UniFormer) and the model does its thing.

Installing it

ComfyUI Manager: search ComfyUI's ControlNet Auxiliary Preprocessors, install, restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/Fannovel16/comfyui_controlnet_aux
pip install -r requirements.txt

Restart ComfyUI. The OneFormer ADE20K weights (250_16_swin_l_oneformer_ade20k_160k.pth) download from HuggingFace on first run. Heads up: this is a Swin-Large model, so it's a chunky download and heavier to run than UniFormer.

OneFormer vs UniFormer

Straightforward call. UniFormer is lighter and faster; OneFormer is bigger, slower, and more accurate. Prototype with UniFormer if speed matters, switch to OneFormer when segmentation quality is what's holding your result back - especially on scenes with lots of overlapping objects. Both target the same ADE20K color scheme, so they drop into the same seg ControlNet interchangeably.

Where people get burned

The recurring 2026 trap is trying to use segmentation with a modern base that has no seg ControlNet - run this on SD 1.5 or SDXL. Beyond that, the main cost here is speed and VRAM: OneFormer is the heavy option, so if it's dragging, UniFormer probably gets you a good-enough map faster. And the constant across the pack: the segmentation map does nothing until a seg ControlNet is loaded downstream to act on it - the preprocessor only draws the layout, the ControlNet is what enforces it.