Nodes/ComfyUI-LexTools/SegformerNode
ComfyUI Node

SegformerNode

The node that actually understands what's in your image

By SOELexicon·Created 3 years ago·Updated about a year ago· 33
SegformerNode
  • image
  • IMAGE
  • MASK
  • STRING
model_nameenes361/segformer_b2_clothes

SegformerNode is the workhorse of the LexTools pack and the one node in here worth installing the whole thing for. Feed it any image and it doesn't just tell you "there's a person" - it labels every pixel: coat, pants, skin, hair, background. That's semantic segmentation, and it's the difference between guessing where a garment is and knowing exactly where it is.

Where you reach for this in a real workflow is inpainting and outfit changes. The community-standard trick for "change the character's clothes" is to run a SegFormer clothes model, take the garment mask it produces, and feed that mask straight into an inpainting pass. People literally built whole custom packs around segformer_b2_clothes for that job; this node hands you the same masks without the extra install. You can also grab a person-segmentation model and use the output as a quick subject mask for compositing or background replacement.

How it works

SegFormer is a transformers-family segmentation model: it runs the image through a hierarchical encoder and classifies each pixel into one of the classes the model was trained on. What those classes are depends entirely on which checkpoint you pick, and that's the one choice that matters here.

The model_name dropdown lists ten HuggingFace repos, and they are not interchangeable:

  • The clothes models (enes361/segformer_b2_clothes, the default, and the mattmdjaga/ variants) label upper body, lower body, full body, and skin.
  • The -human-parsing models split a person into hair, face, torso, arms, legs and so on.
  • s3nh/SegFormer-b0-person-segmentation gives you a person/background binary.
  • ehsanhallo/segformer-b0-scene-parse-150 does full scene parsing - building, road, tree, sky, all 150 ADE20K classes.

Pick the checkpoint that matches the job. Running a clothes model on a landscape and wondering why every mask is empty is the classic first mistake.

The inputs and outputs that matter

Inputs are just two:

  • image - any IMAGE tensor.
  • model_name - the SegFormer checkpoint. The default clothes model is right for 90% of use cases; switch it when you need person parsing or scene parsing.

Outputs are the whole point:

  • IMAGE - the segmented view, so you can eyeball what the model actually found without trusting it.
  • MASK - the mask for the segment (or segments) you asked about. This is what you wire into SetLatentNoiseMask, an inpainting model, or a compositor.
  • STRING - a text report of the detected segments and their labels, handy when you're scripting.

Install

Install once and every LexTools node comes along:

cd ComfyUI/custom_nodes
git clone https://github.com/SOELexicon/ComfyUI-LexTools

or use ComfyUI Manager and search "ComfyUI-LexTools". The pack needs torch, transformers, pillow, matplotlib, numpy, scipy, huggingface_hub, and torchvision - ComfyUI already ships most of those, so in practice you usually only need transformers if it's missing. Restart ComfyUI and the nodes appear under LexTools/ImageProcessing/Segmentation.

Gotchas

The model downloads from HuggingFace on first run, so you need network and a little patience the first time. A couple of hundred MB per checkpoint. If the run dies with a download error, re-run - huggingface_hub caches partial downloads poorly.

The other recurring theme in this pack's family: device errors. If you see "Expected all tensors to be on same device", the model landed on CPU while the image is on GPU (or the reverse). It usually means a stale transformers - update it via pip install -U transformers and retry. And check the preview output before you trust any mask. SegFormer is good, but it's not magic on weird poses, cropped subjects, or odd lighting.

CategoryLexTools/ImageProcessing/Segmentation

Inputs (2)

NameTypeDefaultDescription
imageIMAGE
model_nameCOMBOenes361/segformer_b2_clothes10 options: enes361/segformer_b2_clothes, mattmdjaga/segformer_b0_clothes, mattmdjaga/segformer_b2_clothes, DiTo97/binarization-segformer-b3, s3nh/SegFormer-b0-person-segmentation, venture361/clothes_segmentation, +4

Outputs (3)

NameTypeDescription
IMAGEIMAGE
MASKMASK
STRINGSTRING