Nodes/ComfyUI-Flux-TryOff/Run TryOn Inference [ComfyUI-Flux-TryOff]
ComfyUI Node

Run TryOn Inference [ComfyUI-Flux-TryOff]

Putting a new shirt on a photo, the CatVTON way

By asutermo·Created 2 years ago·Updated about a year ago· 52
Run TryOn Inference [ComfyUI-Flux-TryOff]
  • image_in
  • garment_in
  • mask_in
  • pipe
  • garment_image
  • tryon_image
width576
height768
num_steps50
guidance_scale30.0
seed42
promptThe pair of images highlights clothing and its styling on a model, high resolution, 4K, 8K; [IMAGE1] Detailed product shot of clothing [IMAGE2] The same clothing is worn by a model in a lifestyle setting.

This is where the try-on actually happens. Feed it a photo of a person, a product shot of a garment, and a mask over the person's current clothing, and out the other end comes the person wearing your garment. No API, no cloud, no key - it runs the CatVTON-flux fine-tune locally inside a FLUX.1-dev-fill pipeline.

How it works (the trick that makes VTON work)

Most people expect virtual try-on to be "inpaint the garment into the masked region." That mostly fails, because the model never sees the actual garment as a condition - it just sees your prompt describing it. CatVTON's trick, which this pack inherited wholesale, is to show the model the garment as an image.

The node builds a side-by-side pair: your garment on the left, the person (with the clothing region masked) on the right. It concatenates them horizontally, runs the fill pipeline on a canvas twice as wide as your width setting, and crops the two halves back apart when it's done. That's why the output is a pair of images, and it's why the default prompt looks like it's describing two pictures at once:

"…[IMAGE1] Detailed product shot of clothing [IMAGE2] The same clothing is worn by a model in a lifestyle setting."

[IMAGE1] and [IMAGE2] are how the model refers to the two halves. This is the same in-context approach the original catvton-flux used, and it's why the wrinkles and fabric texture actually track the reference garment instead of being invented from the prompt.

The inputs that matter

  • image_in - the person. garment_in - the garment on a clean background (flat-lay shots work best). mask_in - a mask covering the person's current clothing; the sample workflows generate it with LayerStyle's SegformerB2ClothesUltra, but any mask you can draw works.
  • pipe - the MODEL output of the FluxFill pipeline loaders. There are two; either works, but the auto-downloading FluxFill Pipeline Loader is the one you want.
  • width / height (default 576×768, multiples of 16) - remember the canvas is 2× width internally, so this is per-half, not total.
  • num_steps (default 50) and guidance_scale (default 30) - the default guidance of 30 is way up there compared to plain FLUX.1-dev's 3.5. Don't "fix" it down to 3.5; this is a distilled fine-tune that wants high guidance. The quantized sample workflows run 20 steps at the same 30.
  • seed - set it to compare runs, randomize to explore. prompt - keep the two-image structure; edit the clothing description, not the framing.

The outputs

Two images, both worth keeping:

  • garment_image - the left half, a re-rendered product shot.
  • tryon_image - the right half, your person wearing the new garment. That's the money output. Wire both into a Preview or Save node.

Install & troubleshooting

Install the pack via ComfyUI Manager (search "ComfyUI-Flux-TryOff") or git clone https://github.com/asutermo/ComfyUI-Flux-TryOff into custom_nodes, then restart. First run downloads the transformer, text encoders, and the tiny TAESD1 VAE automatically.

The failure that actually bites people: the example workflows get mask_in from LayerStyle's SegFormer node, and if the SegFormer model isn't present you get an opaque error about segformer_b2_clothes. LayerStyle installs as a node pack via Manager, but the model file has to be cloned separately:

cd ComfyUI/models
git clone https://huggingface.co/mattmdjaga/segformer_b2_clothes

Beyond that, temper your expectations on speed. The author tested on a 4080 and calls it "quite slow"; on a 4090 you're still waiting minutes per image. If it won't fit in VRAM, add the TryOff Quantizer in 8-bit mode and rerun - that's the recommended setup.

CategoryProcessing

Inputs (10)

NameTypeDefaultDescription
image_inIMAGE
garment_inIMAGE
mask_inMASK
pipeMODEL
widthINT576128–1024
heightINT768128–1024
num_stepsINT501–100
guidance_scaleFLOAT30.01–100
seedINT42
promptSTRINGThe pair of images highlights clothing and its styling on a model, high resolution, 4K, 8K; [IMAGE1] Detailed product shot of clothing [IMAGE2] The same clothing is worn by a model in a lifestyle setting.

Outputs (2)

NameTypeDescription
garment_imageIMAGE
tryon_imageIMAGE