ImageAssistedCFGGuider

ImageAssistedCFGGuider is the "hold this composition" node. From the ComfyUI Extra Samplers pack by Clybius, it outputs a GUIDER that steers the denoising trajectory toward a reference latent image using vector projection, on top of ordinary CFG. If you've wanted img2img's structure retention without img2img's "the old image is baked in" feel, this is the tool.

The mechanism is where it gets interesting, and it's genuinely different from anything stock. Each step, after computing the positive conditioning prediction, the guider projects the reference latent onto the positive conditioning direction, subtracts that projection to get the component of the reference not explained by the prompt, normalizes it back to conditioning scale, and then adds the whole correction to the CFG result weighted by image_cfg. The source does it with torch.linalg.norm projections and a calculate_denoised call - that's the "vector projection onto the CFG" the README mentions. In plain terms: it keeps pulling the output toward the reference, but only along the parts of the reference the prompt isn't already producing, so you get structure preservation instead of image pasting.

You also get image_weighting (flat, linear down, cosine down) to control when the pull is strongest - flat tugs evenly, the two "down" modes fade the reference's influence as sampling progresses. weight_scaling (default 1) is an exponent on that weighting curve.

The inputs that matter:

• latent_image - your reference, as a LATENT. Must be the same size as the diffusion latent (the README calls this out, and it's the #1 footgun).
• image_cfg - strength of the image pull. Default 1.0; 0 disables it.
• cfg - the regular classifier-free-guidance scale. Default 8.
• image_weighting / weight_scaling - shape how the pull fades over the run.
• positive / negative - the normal conditionings.

Output: a GUIDER, wired into a custom sampler node's guider input.

Practical use: VAE-encode a reference image into a latent of the same dimensions as your empty latent, feed it here, and run with a prompt that describes a different subject - the structure should survive while the content changes. That's the demo, and it's genuinely fun. It's also a fiddly node: if the reference latent is a different size, or your resolution is off, the projection math quietly produces garbage.

Install is the shared pack routine - ComfyUI Manager → "ComfyUI Extra Samplers", or:

cd ComfyUI/custom_nodes
git clone https://github.com/Clybius/ComfyUI-Extra-Samplers

then restart. No model downloads; dependency kornia. If you want image-steering plus dynamic CFG in one box, MegaCFGGuider from the same pack does both.