SelfGuidanceSampler

This is the node the whole Self-Guidance pack exists for - an unofficial ComfyUI port of Google's "Diffusion Self-Guidance for Controllable Image Generation" paper (Epstein et al., 2023). Give it a photo, name one object you want bigger or smaller and another you want left alone, and it re-renders the image with just that object resized. No masks, no ControlNet, no second model, no prompt engineering gymnastics. Swap it in where you'd normally drop a KSampler.

One thing to get straight up front: this is img2img, not txt2img. The example workflow loads a photo, resizes it to 1024×1024, VAE-encodes it, and feeds those latents into latent_image. There's no empty-latent path here - no source image, no node.

How it works. This is the clever part, and it's why the paper is called self-guidance. There's no external network steering anything. During sampling the node taps the UNet's cross-attention maps - the same maps that decide which image region attends to which text token - and reads out where "carrot" is, how big it is, its rough shape, and what it looks like. For the first max_guidance_iter steps it compares those readings against what you asked for (size scaled by obj_scale), computes a loss, and pushes gradients of that loss back into the latents. The gradient gets added to the normal noise prediction, so every step steers the image toward the object being the right size while everything else just denoises as usual. The fix_object tokens get the same treatment but with near-zero weights (fix_size_omega 0.2, fix_shape_omega 0.2, fix_app_omega 0.1) - they're read out and locked in place so the onion doesn't drift while the carrot shrinks.

The inputs that actually matter:

• target_object / fix_object (CONDITIONING) - feed both from the pack's CLIPConditioning node, which encodes the object names as token indices. No wiring here, no behavior.
• obj_scale - the resize factor. 0.8 shrinks, 1.3 enlarges; range is 0.7–2.5. This is the knob you came for.
• steps (default 128, max 150) and max_guidance_iter (default 84) - total steps and how many of them carry the gradient guidance. max_guidance_iter must be strictly less than steps or the node refuses to run.
• guidance_scale (7.5) and loss_scale (1.0) - the familiar CFG slider and the strength of the object-loss gradient, respectively.
• size_omega / shape_omega / app_omega (5 / 8 / 1.7) - how hard the size, shape, and appearance constraints are enforced; the fix_* versions do the same for the locked object.
• height and width (both default 1024) - the example resizes the source image to match these; mismatch means a distorted result.
• seed, plus the scheduler block (sampler_scheduler_pairs, beta_start, beta_end, beta_schedule, prediction_type, timestep_spacing, steps_offset) - that whole set is diffusers' standard sampler configuration passed straight through. Defaults (Euler Discrete, scaled_linear, epsilon, leading) match SD1.5/SDXL conventions; leave them alone until you know why you're touching them.

Output is a single latent - into VAEDecode it goes.

Install. ComfyUI Manager, search "self-guidance," or:

cd ComfyUI/custom_nodes
git clone https://github.com/forever22777/comfyui-self-guidance

then restart ComfyUI. It pulls diffusers==0.26.3 and fuzzywuzzy==0.18.0, both pinned - worth knowing if another node in your graph wants a different diffusers.

Expectations, set honestly. Only SD1.5 and SDXL work - the node sniffs the UNet's attention-processor count (32 = SD1.5, 140 = SDXL) and throws NotImplementedError on anything else, so no Flux. It's slow: 128 steps with per-step gradient work plus a second copy of the UNet for the autograd path, so watch your VRAM. And it depends entirely on CLIPConditioning correctly finding your object names in the prompt - if that node errors, fix the prompt, not the sampler. Finally, don't confuse this with Self-Attention Guidance (SAG), an older unrelated technique; most search hits for "self guidance" are SAG, and this is a different animal entirely.