Optimization-Based Style Transfer

Most "give this image the style of that one" tools in ComfyUI go one of two ways: train a LoRA on your style references, or bolt on an image-prompt adapter like IP-Adapter that was pretrained to inject a reference embedding. This node does neither. ADOptimizer is the style-transfer half of ComfyUI-Attention-Distillation, and it works by directly optimizing attention features inside the diffusion model at generation time - no training data, no adapter weights, just your two images and a few hundred optimization steps. It's the ComfyUI implementation of a CVPR 2025 paper (Zhou, Gao, Chen, Huang), and on Reddit at launch someone accurately pegged it as sitting in similar territory to StyleAligned - a shared-attention trick for style consistency - but pushed further into an actual optimization loop.

How it actually works

Feed it a content image (the structure/subject you want to keep) and a style image (the look you want applied), and it runs a gradient-based optimization - you can see this directly in the lr (learning rate) and steps parameters, which don't exist in a normal ComfyUI sampler. This isn't denoising a fixed number of steps on a schedule; it's iteratively adjusting attention features so the output's visual character converges toward the style reference while content_weight holds onto the input's structure. That's also why default steps is 200 - way beyond the 20-50 you'd use for a normal generation, because each step is a small optimization nudge, not a denoising jump.

Inputs that matter

• distiller - the DISTILLER from Load Distiller. Required, and it determines which base model (SD1.5, SDXL, or beta Flux) does the work.
• content / style - your two IMAGE inputs, both required. Route them through Load PIL Image → Resize Image first; matching sizes to your height/width here matters more than it does for a typical sampler.
• content_weight (default 0.25, range 0–10) - the balance knob. Higher preserves more of the content image's structure at the cost of weaker style transfer; lower lets the style dominate. This is the setting worth experimenting with first if your result looks too much like one input and not enough like the other.
• steps (default 200, up to 500) and lr (default 0.05) - the optimization budget and step size. More steps generally means a more thorough style transfer, at a direct cost in generation time.
• height / width (default 512 each) and seed (default 2025) round out the rest.

Output is a single IMAGE.

Installing it

ComfyUI Manager - search "ComfyUI-Attention-Distillation" and install.

Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/zichongc/ComfyUI-Attention-Distillation
cd ComfyUI-Attention-Distillation
pip install -r requirements.txt

Restart ComfyUI, then set up the model your distiller will point at - see Load Distiller for the diffusers-format download details (this isn't a drop-a-.safetensors-in-a-folder situation).

Common issues

It's slow, and VRAM climbs faster than you'd expect. This is the honest tradeoff for skipping training entirely: 200-500 steps of gradient-based optimization means backpropagating through attention layers repeatedly, which holds far more in memory than a normal forward-pass sampler running the same step count. If you're OOMing, drop height/width before you drop steps - resolution scales memory faster than step count does here.

Result looks like a blend that favors one image too heavily. That's content_weight, and it's the first thing to touch. Push it down if the output looks too close to the content image and barely styled; push it up if the subject's structure is dissolving into the style reference.

Content and style images at very different sizes or aspect ratios. Normalize both through Resize Image to the same height/width you've set here before running. This pack doesn't hand-hold you through mismatched inputs.

Flux results feel less reliable than SD1.5. The author marks Flux support beta in the pack's changelog; SD1.5 is the best-tested path if you're troubleshooting whether it's your settings or the model.