IP-Comp Concept Merge

This is the payoff node of ComfyUI-IP-Composer. Where the other nodes in this pack build or load the raw material, IP-Comp Concept Merge is the thing that actually does the trick: it takes two image embeddings and a concept subspace, and swaps one image's "concept" for the other's. Give it a giraffe, a zebra, and the fur subspace, and the output embedding describes the giraffe with the zebra's fur. No training, no text prompt fighting for control - a few hundred milliseconds of linear algebra.

The name is a bit understated. Everything interesting about IP-Composer (the paper this pack ports, "Semantic Composition of Visual Concepts") happens inside this one node.

How it works

The mechanics are deceptively simple, and they're right there in the source:

P = concept_subspace["concept_subspace"][:rank, :]
P = P.T @ P  # concept projection matrix
e_mix = ref_embed - ref_embed @ P + concept_embed @ P

A concept subspace is a set of directions in CLIP embedding space that capture one concept - "fur", "outfit", "time of day". P is the projection matrix onto those directions. So the formula reads: take the reference image's embedding, strip out its component along the concept directions, then add the concept image's component along those same directions. Everything else about the reference embedding is left untouched. That's the whole IP-Composer idea in one line, and it's why it feels surgical compared to IP-Adapter weight sliders.

The inputs that matter

• ref_embed - the embedding of the image you're editing (the giraffe). Wire this from an IPAdapterEncoder node in ComfyUI_IPAdapter_plus.
• concept_embed - the embedding of the image that has the concept you want (the zebra). Same source.
• concept_subspace - the concept itself, from IP-Comp Load Concept Subspace or IP-Comp Concept Subspace.
• rank - how many of the subspace's directions to use. Default -1 means "use whatever rank was baked into the subspace when it was built." Lower it to make the edit narrower and weaker; raise it to let more of the concept through. -1 is a fine place to start.

The single output is an EMBEDS tensor, and it wires straight into IPAdapterEmbeds (again from ComfyUI_IPAdapter_plus), which feeds the model during sampling. That's the whole chain: encode two images, merge their embeddings here, and let IP-Adapter do the drawing.

Installing it

The usual ComfyUI two-step:

cd ComfyUI/custom_nodes
git clone https://github.com/godmt/ComfyUI-IP-Composer

then restart ComfyUI (or grab it from ComfyUI Manager by searching "ComfyUI-IP-Composer"). The one Python dependency is open-clip-torch, which Manager installs for you. You'll also want ComfyUI_IPAdapter_plus for the encoder nodes - this pack doesn't ship those.

Where people get burned

• Model mismatch. The subspace and both embeddings must live in the same CLIP vision space. If the subspace was built with ViT-H and your IPAdapterEncoder uses a different CLIP vision model, the merge silently produces mush. Keep everything on ViT-H-14.
• Mismatched input images. The node doesn't care what your images are, which is the feature - but it's also easy to swap ref and concept and wonder why the edit is backwards. ref is the thing you keep, concept is the thing you copy from.
• Rank over-tuning. Cranking rank way up can start dragging in directions that aren't really "fur." If an edit gets noisy, that's usually the first knob to pull back.

The pack is young and this node has essentially no community footprint yet, so the version to trust is the example workflow in the repo - copy that, then swap in your own images and concepts.