UniFormer Segmentor

UniFormer turns a photo into a color-coded map where every pixel is labeled by what it is - sky one color, building another, road, grass, person, each its own flat shade. That segmentation map is what you feed a segmentation ControlNet, and the payoff is control over what goes where without dictating how any of it looks. Sky here, buildings there, a person in this spot - then the prompt decides the style, the weather, the architecture. It's the preprocessor for locking down scene layout while leaving appearance wide open.

How it works

UniFormer is a semantic segmentation model trained on the ADE20K label set (150-ish categories: sky, tree, building, person, car, and so on). It classifies every pixel and paints it the standard color for its class, producing that characteristic flat, blocky map. You wire that into a seg ControlNet (control_v11p_sd15_seg / control_sd15_seg on SD 1.5), and the model generates a new image that honors the regions - a "person"-colored blob becomes a person, a "sky"-colored region becomes sky - while your prompt fills in every detail.

One structural reality to know going in: segmentation is one of the conditions that never got rebuilt for the modern bases. Flux, Qwen-Image, and Z-Image unions cover canny, depth, pose, and edges - but not segmentation or normal maps. So seg conditioning lives in the SD 1.5 / SDXL world (SDXL's xinsir union includes it). If your generator is a 2025-26 model, this preprocessor doesn't have a matching ControlNet to plug into on that base.

The inputs that matter

This one is refreshingly simple. It takes an image and returns an IMAGE - the colored segmentation map - for your ControlNet Apply node. The only knob is:

• resolution (default 512) - the working size. Match it to your render. That's it; there's nothing to tune on the segmentation itself.

Installing it

ComfyUI Manager: search ComfyUI's ControlNet Auxiliary Preprocessors, install, restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/Fannovel16/comfyui_controlnet_aux
pip install -r requirements.txt

Restart ComfyUI. The UniFormer weights (upernet_global_small.pth) download from HuggingFace on first run.

UniFormer vs OneFormer

The pack ships two segmentors and it's worth knowing the difference. UniFormer is the lighter, faster one - the classic segmentation preprocessor from the original ControlNet days. OneFormer (the ADE20K Segmentor node) is heavier and generally more accurate, especially on cluttered scenes. If UniFormer's map is muddy or mislabeling things, OneFormer is the upgrade. Also note SemSegPreprocessor in this same pack is just a legacy alias for UniFormer - same model, same output, kept around for old workflows.

Where people get burned

The main gotcha in 2026 is architecture mismatch: people grab a segmentation preprocessor expecting it to work with their shiny Flux or Z-Image setup, and there's simply no seg ControlNet for those bases. Run seg conditioning on SD 1.5 or SDXL. Beyond that, if the segmentation itself is unreliable on your image, try OneFormer before giving up. And the pack-wide rule: the segmentation map is inert without a seg ControlNet loaded downstream - the preprocessor only draws the map, the ControlNet is what makes the model obey it.