IP2P Conditioning Advanced

Instruct-pix2pix-style checkpoints don't just take a text prompt - they take a source image concatenated into the conditioning too, which is how they know what they're editing. IC-Light is actually built the same way, on the same instruct-pix2pix-style conditioning mechanism, even though it's a relighting model rather than a general editor. If you've ever wired up ComfyUI's built-in IP2P conditioning node and watched it re-run a VAE encode on your source image every time you tweak the prompt, this node is the fix. It works directly on latents instead of pixels, so the expensive encode only happens once.

What it's actually doing

The vanilla approach takes pixels, VAE-encodes them, and bakes the result into your conditioning as a fixed step in the graph. Every prompt tweak re-triggers that encode even though the source image hasn't changed. IP2P Conditioning Advanced sidesteps that: you hand it latents you've already encoded, and it builds the conditioning from those. Swap prompts all day and the encode step never re-runs.

It also splits the source into two separately-scaled latents - original and new - rather than one fixed image. That maps onto how instruct-pix2pix's guidance actually works: these checkpoints are trained with two separate guidance signals, one for how much to follow the text instruction and one for how much to hold onto the source image, and sampling with both requires more than a single conditioning term. That's the practical reason you get two conditioning outputs back instead of one.

Inputs and outputs that matter

• original and new - the two source latents, each with its own scale (original_scale, new_scale, both defaulting to 1, range 0.01–100). Push original_scale up to hang onto more of the source; push new_scale up to weight whatever the "new" latent represents more heavily.
• positive / negative - your usual CLIP text conditioning, passed through and folded in.

Outputs: cond1 and cond2 (the split positive conditioning - wire both into wherever your sampling setup expects the two IP2P guidance terms), negative (passed through unchanged), and latent - the combined starting latent for your KSampler.

Installing it

Through ComfyUI Manager, search ComfyUI Image Filters (repo name ComfyUI-Image-Filters) and install, or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/spacepxl/ComfyUI-Image-Filters

Restart ComfyUI. No model downloads - it's pure latent math - but the pack leans on opencv for its image nodes, and if another custom-node pack already installed a conflicting opencv build you'll get import errors on startup. The maintainer ships import_error_install.bat for exactly that: it strips every opencv variant and reinstalls all four in matching versions. On Linux or by hand, the simpler fix is uninstalling the opencv-* packages and installing just opencv-contrib-python, which covers everything this pack needs.

Where people get stuck

This node only makes sense with an instruct-pix2pix-family checkpoint - the original IP2P model, IC-Light, or a finetune trained the same way. Feed it into a plain SD1.5 or SDXL checkpoint and you're not going to see anything resembling instruction-following; the checkpoint was never trained to read a concatenated-latent conditioning that way.

Second trap: original and new need to be the same latent shape. Mismatched resolutions between the two will throw a shape error at sample time, not at the node itself, which makes it a confusing one to debug if you don't already suspect the conditioning step.

And don't be surprised by getting two conditioning outputs instead of one - that's not a mistake in your graph, it's the two-term guidance IP2P checkpoints expect. If your target sampling setup only wants a single positive conditioning, you'll need to combine cond1 and cond2 rather than picking one and discarding the other.