ControlNet Preprocess Seg

The segmentation preprocessor answers a different question than the other ControlNet conditions. Canny and MLSD give you edges, pose gives you a skeleton - but seg gives you a semantic map: every pixel labeled as sky, building, person, road, grass, and so on. It's the "what goes where" condition, and it's the one you reach for when you want a specific scene layout without caring about the exact edges. This node runs that segmentation on Runware's cloud and hands you the map as an IMAGE.

The practical difference from a sketch: you don't draw anything. You feed it a reference image - a street photo, a room, a cityscape - and it extracts the layout as labeled regions. The model then regenerates that arrangement: a person standing on a sidewalk in front of a building stays arranged that way, even if the building, the person and the light all come out completely different. That's why seg is the standard for scene composition control and for "keep the geography, change the everything-else."

The inputs

Required: image (IMAGE) - the reference whose layout you want to preserve.

Optional: outputFormat (JPG default, PNG/WEBP) and the ttl group. One input, one job. Output is a native image (IMAGE): the color-coded semantic map.

A tip buried in that default: the map is color-coded, and if you're chaining it into a ControlNet that cares about exact class colors, PNG avoids the compression artifacts JPG can smear into thin region boundaries. For most workflows JPG is fine; for tight layout adherence, PNG is cheap insurance.

How it works

Pack-standard: your image goes up as a PNG data URI, semantic segmentation runs on Runware's GPUs, and the labeled map comes back into your graph. Then it's the usual ControlNet chain - condition map into the Runware ControlNet builder, builder into a model node, weight as the adherence dial. The preprocess node is the front half only; on its own it just draws a fancy map.

Install and gotchas

Install the pack once - ComfyUI Manager, search Runware - or git clone https://github.com/Runware/ComfyUI-Runware into custom_nodes + pip install -r requirements.txt, restart, and set a Runware API key (Settings → Runware API key, RUNWARE_API_KEY, or runware auth login).

The honest gotchas: seg holds arrangement, not appearance - expect the model to invent new textures, lighting and even new buildings inside the labeled regions, and don't fight that; it's the feature. And it's a paid chain per generation, so pick your reference image with intent. For "same scene, new everything," seg is the cleanest tool in the ControlNet drawer.