CatVTON Node

Virtual try-on has a reputation for being fiddly - pose estimators, DensePose, six models and a prayer. This node is the opposite. It's a single diffusion node that takes a person photo, a garment photo, and a mask of the region to swap, and hands you back the person wearing the garment. That's it. It's the heavy-lifting core of shadowcz007's comfyui-try-on pack (from the Mixlab family), and honestly the reason the pack exists.

The name is a hint: it wraps CatVTON, a 2024 try-on method, but don't let the "wrapper" reputation scare you. The pack ships its own copy of the CatVTON pipeline, and the scariest part of the original - detectron2 / DensePose install hell - is dead code here. No detectron2, no API, no key, no text prompt.

How it works

Under the hood it loads an SD1.5 inpainting UNet, the classic sd-vae-ft-mse VAE, and a CatVTON attention checkpoint. The clever bit: instead of conditioning on a text prompt, it swaps the UNet's cross-attention for "skip" attention and concatenates the garment's latents next to the masked person's latents, so the reference image is the conditioning. Then it DDIM-samples at low CFG and returns the composite. Everything gets padded to a consistent 768×1024 portrait internally, so your input resolution barely matters.

The inputs that matter

• image - your person photo.
• mask - the region that gets redrawn. White = replace. This is where the pack's other two nodes (Fashion Cloth Mask / Fashion Cloth Mask 2) come in, or any mask you draw yourself.
• refer_image - the garment, ideally flat and straight-on against a plain background. The model copies its texture into the mask region.
• mask_grow (default 25) - dilates the mask so the new garment has room to sit naturally instead of hugging a tight crop. Negative values shrink it.
• steps (40), cfg (2.5), seed - the usual diffusion dials. CatVTON genuinely likes a low CFG; don't crank it.
• mixed_precision - fp16 is the default and the right call on most cards. Drop to bf16 (newer GPUs) or fp32 if you hit NaNs or artifacts.
• attn_ckpt_version - mix / vitonhd / dresscode, matching which training the attention weights came from. mix is the general-purpose one; leave it there unless you know the checkpoint you downloaded is a specific variant.
• device - auto or cpu. cpu technically works; bring a book.

Output: a single image - the person wearing your garment.

Installing it

ComfyUI Manager - search comfyui-try-on - or:

cd ComfyUI/custom_nodes
git clone https://github.com/shadowcz007/comfyui-try-on

then restart. Two gotchas that will bite you:

1. There is no requirements.txt in this pack. Manager installs the code and nothing else. You need transformers, diffusers, accelerate, and scipy in ComfyUI's Python yourself if they aren't already.
2. The README's model story is thin. It points at a Baidu Netdisk link (extraction code MAI0) for the CatVTON weights → put them in ComfyUI/models/catvton/. But the node also expects a stable-diffusion-inpainting/ folder and an sd-vae-ft-mse/ folder in the same directory - the README never mentions either. Grab them from Hugging Face (runwayml/stable-diffusion-inpainting and stabilityai/sd-vae-ft-mse, diffusers format).

If models/catvton doesn't exist at all, the node tries to snapshot_download its path as a HuggingFace repo id - you get a baffling "repo not found" style error. That's your "model not in the right place" signal.

Common issues

• Garment looks soft or logos pixelate. That's CatVTON's known weakness, not your setup. Upscaling the result makes garment texture worse, so people tend to run a FaceDetailer-style pass after instead.
• The output redraws the whole person → your mask is wrong, see the two Fashion Cloth Mask articles.
• NaN / OOM in fp16 → switch mixed_precision to bf16 or fp32. SD1.5-based, so it fits on most consumer GPUs at fp16.

The whole pack is built around this node - worth the setup for anyone doing product or fashion shots.