VAE Encode ArgMax

Here's a thing almost nobody thinks about: a VAE encode isn't necessarily deterministic. The SD 1.5 VAE is an AutoencoderKL, and in that architecture "encoding" can include a reparametrization-trick sample - a small random draw from the learned latent distribution, on top of the mean. Run the same image through VAEEncode twice and you get two latents that are almost but not quite identical.

Usually that noise is invisible. But IC-Light conditions on its VAE-encoded inputs in a very direct way - the encoded foreground is the lighting context, fed straight into the UNet as extra channels. If that conditioning re-rolls a sample of noise on every run, your light direction wobbles, your results drift between identical seeds, and a sharp light map gets a fuzz of randomness smeared over it. That's the problem VAEEncodeArgMax exists to kill.

What it does

It's a subclass of ComfyUI's built-in VAEEncode that, in its own words, sets regularization.sample = False to "obtain mode of distribution." Concretely: it flips the VAE's sampling flag off, runs the normal encode, and flips it back on so nothing downstream is affected. You get the mode of the latent distribution instead of a random draw - the mean, the most likely latent. Same pixels in, same latent out, every single time.

The pack's example workflows use it on both IC-Light conditioning paths: the foreground you encode into c_concat, and - in the "preserve original color" example - a second encode of the subject that gets placed into the latent space to keep the object's true colors guiding the generation. Deterministic encodes are what make that trick stable.

Inputs and output

A straight drop-in replacement for VAEEncode, with identical wiring:

• pixels (IMAGE) - the image to encode. For IC-Light conditioning this is your grey-filled foreground (run it through ICLightApplyMaskGrey first).
• vae (VAE) - the SD 1.5 VAE from your CheckpointLoaderSimple.

Output: a single LATENT, ready for the c_concat input of ICLightAppply or for blending into a KSampler's latent image.

Not IC-Light-only

Because it's just "VAE encode without the sampling noise," you can swap it into any SD 1.5 pipeline where you want reproducible latents - img2img strength experiments, or any chain where you re-encode the same input repeatedly and want it to stay put. Just don't expect it to work on every VAE.

Gotchas

• It only supports AutoencoderKL. The code asserts this, and the error message says it plainly: "ArgMax only supported for AutoencoderKL." SD 1.5's VAE is one, so you're fine here; if you ever wire in a different VAE architecture you'll get an assertion failure, not a silent wrong result.
• It changes nothing about IC-Light itself. It's the conditioning path that gets more stable, not the relight quality - the model is the same frozen IC-Light V1 (see the KB's IC-Light panel for why that's both its strength and its ceiling).
• Don't use it for the decode side. This is an encoder-only tweak; decoding is untouched.

Install

It ships with the huchenlei/ComfyUI-IC-Light-Native pack, which has no pip dependencies - it's pure ComfyUI-native code. Install the pack, restart, and the node appears:

cd ComfyUI/custom_nodes
git clone https://github.com/huchenlei/ComfyUI-IC-Light-Native

If you're already running an IC-Light workflow and getting subtly inconsistent relights between identical runs, replacing your plain VAEEncode with this is a five-second experiment. It won't fix a bad prompt or a wrong model - but it will stop your conditioning from rolling dice every run.