Concept Segmentation

What it is

Point this at an image and a comma-separated list of concepts and it labels every pixel by which concept it belongs to - zero-shot semantic segmentation, driven by the diffusion model's own attention instead of a dedicated segmentation network. No boxes to draw, no clicks, no separate SAM checkpoint to download. If the diffusion model can describe a concept in text, this node can (in theory) find it in the image.

How it compares to the usual tools

The ecosystem's standard route is a trained segmenter - SAM, BiRefNet, u2net (the KB's background-removal essay covers that landscape in detail). Those are the right tool for clean cutouts with sharp edges. This node is the different path: it leans on the DiT's concept attention, so it costs you no extra model and generalizes to any concept you can name, but the masks are only ever as precise as the attention behind them - fuzzy, approximate, definitely not BiRefNet hair-level. Great for a quick "where's the person vs the sky" pass, wrong tool for production matting.

Inputs

Three of them:

• concept_maps (CONCEPT_MAPS) - the bundle from ConceptAttentionNode.
• image (IMAGE) - the original image, used to build the colored result.
• concepts (STRING, multiline, comma-separated) - the list to segment by, e.g. woman, cat, white, lines, cane.

Outputs

• segmentation_mask (MASK) - the labeled mask, where different concepts get different values. Wire it into masking, conditioning, or compositing.
• segmented_image (IMAGE) - the original image with the segmentation applied as a colored overlay, ready to save.

The rough edge you need to know

Reading the shipped source, the segmentation loop iterates the concepts string without splitting it on commas the way ConceptAttentionNode does. In practice that means a normal comma-separated list tends to come back as a blank mask, because the node is matching whole words against single characters. This is the roughest node in the pack - genuinely unfinished. The reliable workaround: pull individual masks out of ConceptSaliencyMapNode per concept and combine them yourself with standard mask-compositing nodes. And every caveat from the main node applies here too - exact concept names, DiT-only models, empty-map fallbacks.

How to install

Same pack as the rest. In ComfyUI Manager, search "Concept-Diffusion", or:

cd ComfyUI/custom_nodes
git clone https://github.com/Junst/ComfyUI-Concept-Diffusion
cd ComfyUI-Concept-Diffusion
pip install -r requirements.txt

Restart ComfyUI. No model downloads - it runs off the concept maps already in your graph, which is the entire appeal: segmentation without ever loading a segmentation model.

When you'd reach for it

When you want a text-driven label map for a scene and don't want to add another model to your setup. It's an idea worth playing with - attention-based localization is a genuinely cool trick - just budget some time for the rough edges and verify that first mask before you build on it.