Latent Interposer

Ever generated an image with SD1.5 and wished you could hand its composition to SDXL - or run a Flux image through an SDXL img2img pass - without it looking like a melted watercolor? The usual route is a VAE decode, an upscale, a VAE encode, and a prayer. Latent Interposer skips all of it. It's a tiny neural network that translates a latent from one model family's latent space directly into another's, in-place, no pixels involved.

The name is not a joke about an API or a key - there is none. It's a small conv net that city96 trained to map between latent spaces, and it's one of those "of course this exists" nodes you keep forgetting you have until you need it for exactly one thing.

How it works

Every Stable Diffusion family uses a VAE that compresses images into a latent space - but those spaces aren't compatible. SD1.5 and SDXL both use 4 latent channels at 8x compression, so they're close; Flux and SD3 jumped to 16 channels, and Stable Cascade's Stage A/B is its own thing. Shove an SDXL latent into a Flux sampler and you get noise, not an image.

The interposer is a small conv network - 12 residual blocks with BatchNorm, 64 hidden channels - trained to convert between specific pairs: v1 (SD1.x), xl (SDXL), v3 (SD3), fx (Flux), and ca (Stable Cascade). It projects channel counts (16→4 for Flux→SDXL, say) and learns the actual statistical mapping between the two latent distributions. Version 4.0's training used a round-trip loss - run the source latent through the converter, then back through the reverse model, and check it matches - which is why the results hold composition instead of just color.

One detail the README won't tell you: the node runs in FP32 on CPU, not your GPU. It's a tiny net so it's basically instant on any modern machine either way.

The inputs that matter

Only three inputs, and you'll set all of them:

• samples - the LATENT you're converting.
• latent_src - which family your latent came from: v1, xl, v3, fx, or ca.
• latent_dst - the family you're converting to. Note the dropdown only offers v1, xl, and v3. That's not a bug: the availability matrix has no models that convert into Flux or Cascade - those are source-only. Flux→SDXL works great; SDXL→Flux does not exist.

Output is a single LATENT, wired straight into whatever sampler you're targeting. If latent_src == latent_dst, the node just returns your samples untouched - a free pass-through you'll probably never hit, but good to know it doesn't crash.

Installing it

Easiest way: ComfyUI Manager, search "Latent-Interposer" (pack title), install, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/city96/SD-Latent-Interposer

Or skip the repo entirely and drop the single comfy_latent_interposer.py file into custom_nodes. The node pulls weights from Hugging Face automatically on first use, so it needs huggingface-hub (install it in your venv with pip install huggingface-hub if ComfyUI complains). Offline or flaky connection? Clone the weights into ComfyUI/custom_nodes/SD-Latent-Interposer/models and it'll prefer those:

git clone https://huggingface.co/city96/SD-Latent-Interposer custom_nodes/SD-Latent-Interposer/models

No other dependencies - this pack is refreshingly self-contained, unlike some custom nodes that drag in half of PyAV.

Where it fits in a workflow

You use it exactly where you'd otherwise stack a VAE Decode → (upscale) → VAE Encode. Classic case: generate a base composition with SD1.5 at 512, feed it through the interposer into an SDXL sampler at higher denoise to refine. A reddit thread that's basically the canonical writeup on this ("more direct converter from sd latent to sdxl latent?") confirms the practical difference: going through pixels makes the first model's VAE and style bleed into the result; staying in latent space gives the second model more freedom, with differences at the small-grained detail level.

Set your denoise to hide artifacts while keeping the composition - the author's own advice, and it matters. The interposer is learned and lossy, not a perfect inverse. Start around 0.6–0.7 and tune.

Troubleshooting

• "No model exists for this conversion!" - you picked a source/destination pair the matrix doesn't cover. You can't go into Flux or Cascade; pick a different latent_dst.
• First run seems stuck - it's downloading the weights from HF. Watch the console; if it errors, check huggingface-hub is installed.
• Color/hue shift on output - a known limitation the author is upfront about. Tune denoise, or accept it - a decode/encode round trip has its own losses, just different ones.