LC VAE Merge

The idea

A diffusion model was trained against one VAE's latents. That pins the encoder: change it and the latents your model produces no longer mean what it was trained to mean. The decoder is the free half. So you can take one VAE's decoder and blend it into another's, keep the base's encoder, and the result still fits every model that was made for the base.

That's the whole trick here. vae_a is the base and its encoder is always kept. vae_b contributes the decoder - typically a decoder finetune of the same VAE, the kind of thing that trades a little texture for sharper edges or warmer colour.

How it works

ComfyUI's loader has already normalised layout by the time the node sees the two VAEs, so diffusers-style files (up_blocks / mid_block) merge with original-layout ones - no conversion step. The node walks the state dict, sorts every decoder key into a region, and linear-interpolates each region between the two by its own slider.

The regions go coarse to fine, and the tooltips are worth reading because they tell you the order to think in:

• start - decoder input and middle: overall tone and colour.
• coarse - first upsampling stage: large shapes.
• medium - second stage.
• fine - third stage.
• finest - last stage and the output layer: fine texture, sharpness, the last colour mapping.

Every slider is 0 to 1, where 0 is all vae_a and 1 is all vae_b. Output is vae, plus a report.

One mechanical detail you don't have to think about but would otherwise assume wrong: the output layer always travels with the finest stage, never separately. The author tested taking one VAE's output layer on top of the other's finest stage on Wan / Qwen-Image style decoders, and it falls apart into a grid pattern and a green cast.

Watching the report

The report carries a per-region distance - how different the two decoders are, as a percentage, where 0% is identical - plus whether the encoders match. That number is the whole safety rail:

Under about 10%, blend freely. Over 15% with a slider set somewhere in the middle, and the node warns that one decoder was retrained rather than finetuned and you'll get oversaturated colour or noise. Same family matters too: Krea 2 and Anima both use the Qwen-Image VAE, and the Qwen-Image and Wan VAE pairs sit 30-70% apart - blending those is not a small judgement call, it's a different model's latent space.

It also flags big jumps between neighbouring regions. Flux VAEs take a hard jump fine. The Wan / Qwen-Image lineage (Krea 2, Anima) doesn't - ramp it, something like 0, 0.25, 0.5, 0.75, 1 across the five sliders instead of going 0,0,0,1,1.

Install and then what

cd ComfyUI/custom_nodes
git clone https://github.com/lonecatone23/ComfyUI_LC_ModelBuilder_nodes

Restart. Nothing to install - this is core ComfyUI plus torch.

The merged VAE lives in the graph only, so save it with LC VAE Save (same pack, into models/vae) before you move on, or you'll be rebuilding the merge every session. Then compare the result against both parents with LC VAE Compare on the same latents - that's the honest way to find out whether the blend was worth it.

Where people get burned

Blending two VAEs that are far apart is the classic failure, and it doesn't announce itself as an error. You get output, it's just wrong: neon colour, or a sprinkling of noise in flat areas. If the report's distance number is over 10%, use 0 or 1 per region instead of a middle value - pick a decoder, don't average two.

The other one is expecting a VAE merge to fix a model. It can't move the latent space, so it won't make a mismatched VAE work with a model trained for a different one, and no slider setting will.