Apply IPAdapter SD3 Model

Every IP-Adapter workflow is really two nodes: one that loads the adapter, and one that shoves a reference image into the model. ApplyIPAdapterSD3 is the second one - the part where the magic happens. It takes your SD3.5 Large model, a SigLIP-encoded reference image, and the loaded InstantX adapter, and returns a patched model that will generate toward whatever your reference looks like. Style transfer, a recurring subject, a vibe - no LoRA training required, exactly like IP-Adapter has worked on SD 1.5 and SDXL for years.

Worth saying plainly: SD3.5's ecosystem is thin. The community's own verdict is that auxiliary tools like IP-Adapter "arrived late and have limited coverage" on the SD3 family, so this pack (InstantX's official weights, wrapped by Slickytail after InstantX left ComfyUI support to the community) is one of the few image-conditioning routes on the model at all. If you're on SD3.5 Large and you want a reference image in the loop, this is it.

How it works

The interesting bit is that this is a timestep-aware adapter, which is what makes it SD3-specific rather than a straight port of the old Tencent ones. Here's the sequence:

1. Your CLIPVisionEncode node produces a CLIP_VISION_OUTPUT; this node pulls the penultimate hidden states from it - the image embedding.
2. It appends a zero vector to stand in for the uncond side, so classifier-free guidance still has something to steer away from. (The code also only passes the conditional branch through the resampler, a nice efficiency touch.)
3. It hooks the diffusion model's forward pass. During denoising, it reads the current timestep, feeds it to the TimeResampler, and injects the resulting image tokens into the attention of every MMDiT joint block - but only inside your start_percent / end_percent window.

So the reference image isn't bolted on once; it's injected step-by-step, with the adapter able to modulate itself by how far into generation you are.

The inputs that matter

Five inputs, and honestly you'll only touch three of them most days:

• model - your SD3.5 Large checkpoint from a CheckpointLoaderSimple. The node clones it and returns the patched clone.
• image_embed - a CLIP_VISION_OUTPUT from CLIPVisionEncode, which needs siglip_vision_patch14_384.safetensors loaded via CLIPVisionLoader, fed by a LoadImage. Don't reuse the CLIP-ViT-H vision model from your SD 1.5 IP-Adapter workflows; this adapter wants the SigLIP 384 encoder.
• weight - the strength of the image conditioning, from -1 to 5. Default is 1.0, but the README's explicit recommendation is 0.5, and that's not a style preference: this adapter was trained on SD3.5 Large, and at 1.0 the reference tends to stomp all over your prompt. Dial it down.
• start_percent / end_percent - the denoising window (0 to 1). Keep the full 0→1 range unless you're doing the classic trick of ending the adapter early to preserve detail.

The MODEL output wires straight into a KSampler. Your reference goes in the LoadImage, your text goes in via the usual SD3.5 triple CLIP (clip_g, clip_l, t5xxl_fp16).

Setup and things that bite

Installing the pack: search ComfyUI-InstantX-IPAdapter-SD3 in ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/Slickytail/ComfyUI-InstantX-IPAdapter-SD3
# restart ComfyUI

Then grab ip-adapter.bin (from InstantX/SD3.5-Large-IP-Adapter) into models/ipadapter and siglip_vision_patch14_384.safetensors (from Comfy-Org/sigclip_vision_384) into models/clip_vision. Dependencies are just torch, einops, diffusers - nothing to fight.

Where people actually get burned:

• It only works on SD3.5 Large and Large Turbo, and the adapter is hardcoded for Large's architecture. Medium? No. And on Turbo the fidelity is measurably worse, because InstantX trained on Large - the README warns you, and the warning is accurate.
• Nothing changes at weight 0.5? Check your CLIPVisionEncode is actually the SigLIP model and that an image is loaded. A stale vision model quietly makes the conditioning meaningless.
• Forgetting the base model is heavy. SD3.5 Large is an 8B MMDiT - you want ~24GB VRAM before the adapter's overhead lands on top.
• One adapter per chain. The README's TODO list admits multiple adapters can't stack yet - they overwrite each other. Plan around that.

The one genuinely fiddly setting is weight: start at 0.5, nudge up if the reference isn't showing through, down if your prompt stops mattering. That's the whole game, and it's a good game while SD3.5 is still worth running.