ControlNet Pre (+Model/ControlNet): Segmentation

Segmentation conditioning gives the model a layout - this region is sky, this region is a person, this region is a building - without saying anything about what any of it should look like. Colors in the output map carry no meaning of their own, only which pixels belong together. That's genuinely powerful for controlling composition while leaving the model free to invent appearance. Before you build a workflow around this node, though, there's a compatibility question worth answering first: does a segmentation ControlNet even exist for the base model you're using?

Check this before anything else

Segmentation is one of the conditions the modern ControlNet ecosystem quietly dropped. SDXL's xinsir union still covers it - it's one of twelve conditions in that single checkpoint. But none of the newer union models built for Flux, Qwen-Image, or Flux 2 include segmentation at all. The one exception on the current generation is Z-Image, where a community member trained a SAM-based segmentation ControlNet from scratch in 2026 for roughly $200 of rented GPU time. So: on SDXL, you're covered. On Z-Image, there's a community model. On Flux or Qwen-Image, there's currently no segmentation ControlNet to feed this node's output into - the preprocessor working perfectly doesn't help if nothing downstream can consume it.

Two backends, and the same weight-file gap as MLSD

backend offers auto, onnx_segformer, and torch_deeplab - two genuinely different segmentation models. SegFormer is the modern transformer-based approach (the default onnx_model filename, segformer_b5_ade20k_640x640.onnx, names both the model size and ADE20K, the standard 150-class scene-segmentation dataset this whole family of models trains on). DeepLab is the older, more established CNN-based architecture; since it's named torch_deeplab rather than referencing an onnx file, it most likely pulls its weights through torchvision's own model zoo automatically on first use, the way most torch_*-prefixed backends in the ComfyUI ecosystem tend to work - which would make it the lower-friction of the two options.

The onnx path has the same undocumented gap the pack's MLSD node has: weights_dir defaults to weights/segmentation and onnx_model defaults to segformer_b5_ade20k_640x640.onnx, but there's no README pointer to where that file actually comes from. If you'd rather not hunt for it, set backend to torch_deeplab and let it fetch its own weights instead.

The rest of the settings

render_style picks how segments get colored: ade20k_palette uses the standard ADE20K color scheme (the one most segmentation ControlNets were trained to expect - start here), random_palette assigns arbitrary colors per detected region, and class_index_gray renders class IDs as grayscale values instead of color. detect_resolution sets the working resolution independent of your output size. background_color and edge_overlay are cosmetic - background fill color and an optional edge outline over the segments - useful for visually checking a map, less relevant to what the ControlNet actually reads.

Inputs and outputs

Required: image, backend, render_style, detect_resolution, weights_dir, onnx_model, background_color, edge_overlay. Optional model/clip/vae/control_net pass straight through to matching outputs, the same convenience wiring every Loaders node in this pack offers. Output: control_image (IMAGE) - feed it to your ControlNet Apply node, once you've confirmed a compatible segmentation checkpoint actually exists for your base model.

Installing it

Search ComfyUI-CN-Pre in ComfyUI Manager, or cd ComfyUI/custom_nodes && git clone https://github.com/OKIE5/ComfyUI-CN-Pre and restart. The pack's README is a one-line stub unrelated to what it actually does, and there's no community track record to lean on - go by the node's own parameters, not the docs.

Where people get burned

The most likely disappointment isn't the node itself - it's discovering, after building the whole preprocessing side of a workflow, that there's no segmentation ControlNet for your current base model. Check that first. After that, the same onnx-weight gap as MLSD applies: if onnx_segformer fails to find its model file, switch to torch_deeplab rather than spending time tracking down a download link the pack never gives you.