Add Cosine A

Plain Add Difference adds a delta everywhere, full stop. Add Cosine A is the opinionated version: it looks at how similar each weight vector in a and b actually is, and scales the addition by that similarity. Weights that point in the same direction get blended in; weights that point in wildly different directions get left mostly alone. It's a "merge only what agrees" node, and it exists because the naive add-difference formula has a failure mode: it smears a change across the whole network even where the two models fundamentally disagree.

How it works

The node computes the cosine similarity between a and b per key, then uses it to drive how much of b leaks into the result. The exact mechanism (from the sd-mecha source): k = 1 − clamp(similarity − alpha, 0, 1), then lerp(a, b, k). So when similarity is high, k is small and a stays dominant; when similarity is low, k climbs toward 1 and b gets a real say. The alpha (default 1.0) sets the threshold for "similar enough to keep." The result stays in weight space and the whole thing is lazy - one key at a time, no full model load.

Inputs:

• a (weight) and b (weight) - two model recipes in weight space.
• alpha - the similarity cutoff. Default 1.0. Lower it and more of b gets through; raise it and the merge gets more conservative.

Output: one recipe (MECHA_RECIPE).

Add Cosine A vs Add Cosine B

There are two flavors of this in the pack and they're nearly identical. Add Cosine A uses plain cosine similarity between the (normalized) weight vectors. Add Cosine B averages that with a magnitude-aware similarity, which is slightly more forgiving of models at different scales. For most merges you won't be able to tell them apart; B is marginally more robust when your two checkpoints were trained at different scales. If you only remember one, remember that the output is per-key, so it's effectively a similarity-weighted mask - a cousin of Train Difference Mask, but simpler and cheaper.

When to reach for it

The best case is a merge where you want the parent's direction to survive: blending two stylistically close models without letting their disagreements drag each other toward mush. It's less aggressive than full masked merging, so it's a good middle step before you go hunting for a hand-tuned mask. It won't fix a merge where the parents are simply incompatible - cosine similarity being near zero just means "leave it alone," not "make it work."

merge_checkpointing works as everywhere: cache this branch on CPU in fp16 once the recipe is stable.

Install & gotchas

ComfyUI Manager → search mecha → "Mecha Merge Node Pack", or:

cd ComfyUI/custom_nodes
git clone https://github.com/ljleb/comfy-mecha.git
pip install -r comfy-mecha/requirements.txt

Restart after. Dependency is sd-mecha==1.1.7.

Feed it weight-space models, not deltas - the similarity math assumes actual weights. And don't expect dramatic differences from plain add-difference on small merges; the payoff shows up on bigger, multi-model recipes where directionality actually varies.