AGSoft Reference to Latent

Flux doesn't do img2img the way SD 1.5 did. Its family of models expects reference images to arrive inside the conditioning as reference_latents, not as a separate latent you blend - which means the "standard" Flux reference-image workflow is a fiddly multi-node dance with the VAE and some conditioning plumbing. AGSoft Reference to Latent compresses that dance into one node: feed it conditioning, a VAE, and up to ten reference images, and it hands back conditioning that Flux can actually consume, plus a base latent and mask. If you've been copy-pasting Flux reference workflows, this is the part that keeps getting re-wired.

How it works

The mechanism is exactly what the name says, and it's grounded in how Flux conditions:

1. Your positive and negative conditioning come in from the CLIP/text-encoder side.
2. Each reference image you connect (image1–image10, chosen by mode) gets encoded into latent space with the vae you provide.
3. The node calls ComfyUI's own conditioning_set_values(..., {"reference_latents": ref_latents}, append=True) on both positive and negative - the canonical way to attach reference latents to conditioning for Flux-family models.

The mode dropdown (1_image through 10_image) controls how many imageX inputs appear - dynamic inputs driven by the frontend, so you only see the sockets you need.

It returns five outputs:

• positive / negative - your conditioning, now carrying reference_latents.
• latent - the first reference image encoded as a LATENT, which serves as the base latent for the sampler (Flux's img2img flow works by sampling from that latent).
• mask - the image1_mask you optionally supplied, auto-scaled to latent size (this is what limits how much of the base latent gets overwritten).
• image - the original first reference image, passed through for convenience (preview, or a second use downstream).

What to wire where

• positive, negative, vae - three required inputs; the VAE must match the model family (Flux's own VAE for Flux workflows - the KB's point about VAE/latent-space compatibility applies: wrong VAE, wrong latent space, garbage out).
• mode - pick how many references; start with 1.
• image1 - your main reference; it becomes both a reference latent and the base latent.
• image1_mask - optional, but this is the lever that makes it "edit this part, keep the rest." Without a mask, the sampler tends to redraw from the whole base latent.

Then take positive/negative and latent into a KSampler. That's the whole integration.

Installing and gotchas

cd ComfyUI/custom_nodes
git clone https://github.com/Art-xmaster/comfyui-AGSoft

Restart (or ComfyUI Manager → "comfyui-AGSoft"). No model downloads - you supply your own VAE.

Two traps trip people up. First, the conditioning only helps if your model expects reference_latents - Flux dev/schnell and their derivatives do; SD 1.5/XL checkpoints don't, and wiring this into one of those just silently does nothing useful. Second, resolution: reference images that aren't at a compatible size get rounded to a valid latent size by the node's own helper, but a wildly-off-size reference can still come out stretched - upscale or crop to your target aspect first. If the output ignores the reference entirely, the mask is your first suspect (too tight, or none supplied).