SegToMask SegToMaskControlnet

What it does

SegToMaskControlnet is the pack's third node, and it exists for a ControlNet conditioning type that time quietly left behind: the semantic segmentation map. It runs the exact same segmentation pass as its sibling SegToMask - same inputs, same pipeline - but instead of handing you a clean mask, it outputs a single IMAGE: the labeled segmentation map, where each region is painted according to its class. That image is a ready-made ControlNet condition.

Why would you want that? Feed a seg map into a ControlNet and the model doesn't just see "a person here" - it sees the whole layout as a painted canvas, so generation respects "sky up top, person in the middle, floor below" without you drawing a single stroke. In the SD 1.5 era this was the standard way to control scene composition: detect the layout from a reference, then regenerate it with a different style or different content.

How to use it

Inputs are the same three-ish you saw on SegToMask: image (what you're segmenting), classes (the multiline list - the only input you'll actually edit), and model + processor wired straight from the pack's PipelineLoader. There's exactly one output, IMAGE, and it goes into a ControlNet Apply node as the conditioning image, alongside a ControlNet that actually speaks seg - SD 1.5's control_v11p_sd15_seg, or the seg mode on an SDXL union. Start at a control weight around 0.7–1.0 to hold the layout, and release the condition once composition locks in (start step 0, end step ~0.5 or so) rather than riding it through the whole sample.

The honest take

Now the honest take, and it's a big one. Segmentation is a golden-era SD 1.5 conditioning. The modern union ControlNets dropped it - seg maps, like QR and reference, were never rebuilt for the post-Flux architectures. And in current ComfyUI the maintained route to a segmentation map is OneFormer or UniFormer from the ControlNet Auxiliary Preprocessors pack: it has a real model picker, gets updated, and covers exactly this job. So this node is a nostalgia piece for vintage SD 1.5 seg workflows. If your exact setup is an old checkpoint with control_v11p_sd15_seg, it'll do the job. If you're on anything modern, skip it and reach for the preprocessors.

Troubleshooting

Troubleshooting, briefly, because the failure modes mirror the rest of this pack:

• Repo is dead. Same story as its siblings - empty README, 404 GitHub page, a couple of installs. Install via Manager (search segment_to_mask_comfyui) or git clone https://github.com/ginlov/segment_to_mask_comfyui into ComfyUI/custom_nodes, then restart. First run downloads the segmentation checkpoint from the HuggingFace Hub.
• Wrong ControlNet, wrong output. A seg map fed into a canny or depth net doesn't error - it just silently conditions on something the model never trained on. Make sure the ControlNet you load actually accepts segmentation maps.
• Blank or empty map - your class names aren't in the model's vocabulary. Recheck spelling and case before anything else.

Bottom line

Bottom line: it's a tidy little seg-map generator, but it's a tool for a workflow that's mostly historical. If you're building a modern graph, the same layout control lives in maintained preprocessors, and this node is only worth wiring up if you're deliberately doing vintage SD 1.5 seg ControlNet.