Paint by Example (simple)

Your prompt is a picture

Most inpainting in ComfyUI is prompt-driven: you mask a region, type "golden retriever," and hope the sampler does the rest. Paint by Example (simple) does the opposite. You mask a region, hand it an example image, and it fills the mask with something that reads like the example - no text prompt, no CLIP text encoder, nothing typed at all.

It's a throwback, and it's worth knowing that before you invest time in it. This is the "Paint by Example" idea from the 2022–23 research wave (Fantasy-Studio's model, the same era as Lama Cleaner's exemplar-inpaint mode), years before IP-Adapter turned image prompting into a standard technique and long before instruction editors like Qwen-Image-Edit made masked editing feel old. It shows its age: results are often rough, and the pack's own README cheerfully admits the output "may not be very similar to the example image." But it's a genuinely different input - reference-by-picture rather than reference-by-prompt - and occasionally it produces something you'd never get from words. If you're deciding between this and a modern inpaint, pick the modern one; if you want a quick, weird, no-prompt experiment, this is the cleanest ComfyUI wrapper around.

How it works

Under the hood it's the diffusers PaintByExamplePipeline running the Fantasy-Studio/Paint-by-Example checkpoint: a Stable-Diffusion-derived UNet fine-tuned for masked inpainting where CLIP image embeddings of your example stand in for text conditioning. You don't download anything by hand - the first time the node runs it pulls the model from Hugging Face automatically, which is a real wait and eats more than 5GB of disk space (the README's own warning). It loads in float16 and picks cuda, then Apple's mps, then cpu, so it runs on Macs too - just slowly.

The inputs that matter

• image - the photo you're editing.
• mask - where to fill. Paint it with a mask editor, or wire in a mask from SAM or a segmentation node.
• example - the reference. This is the actual "prompt."
• seed - default 0. Bump it to explore variations.
• steps - default 30. The usual 20–40 range is fine.

It returns a single IMAGE, which you save or wire into the next node like any generation output. One nice touch: it batches by broadcasting to the longest of image/example/mask, cycling the shorter ones - so you can run one example across a whole batch of images, or several examples against a single image.

Install

ComfyUI Manager: search "paint-by-example" (the pack is titled paint-by-example @ ComfyUI). Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/phyblas/paint-by-example_comfyui

Then restart ComfyUI. Dependencies are just diffusers, torch, torchvision - light, and Manager installs them for you.

Where people get burned

First run downloads the model - budget the wait and the disk space. The bigger gotcha: on a recent transformers, the pipeline can throw at import, and the pack's own fix is to pin an old version:

pip install transformers==4.40.0

That pin is blunt - it can break other custom nodes that want newer transformers - so try it in a throwaway environment before you downgrade your main install. Finally, don't trust the README's node count: it still says the pack ships 3 nodes, but the code registers 5 (there are two mask-fill helpers riding in the same pack). The README just hasn't caught up with the code.