LoRA Add (Rank Aware) (Yogurt Nodes)

Combining two LoRAs is easy when they share a rank - you just add the tensors. It gets ugly when they don't: a rank-128 style LoRA and a rank-8 character LoRA have different-shaped lora_down/lora_up pairs per layer, and you can't add shapes that don't match. LoRA Add (Rank Aware) is this pack's answer to that exact problem. It merges two LoRAs into one, and when the ranks differ it uses SVD to bring them to a common rank before adding.

The "rank aware" in the name is the whole pitch. You feed it two LORA objects plus per-LoRA strength (alpha_a, alpha_b), and it returns a single merged merged_lora you can save or apply. This is the workflow for "I want this character's identity and this painter's style as one LoRA I can load in one place" - the kind of merge people used to do by stacking both LoRAs at the sampler with fiddly strengths.

How it works

The source is worth a quick walk because the details explain why the result is (mostly) sane. First it absorbs each LoRA's alpha into its weights - it scales the down weights by alpha / rank so the merge works on the actual effect rather than the raw weights. Then, for every layer both LoRAs share, it checks ranks. If either differs from the target rank, it computes that layer's effective weight delta (up @ down), runs a truncated SVD, and re-factorizes at the target rank. Only then does it add: alpha_a * down_a + alpha_b * down_b, same for the up weights. Layers only present in B get copied across as-is. It's a weighted sum with a rank-alignment pre-pass - honest math, no magic.

The inputs that matter

• loraA / loraB - the two LoRAs. Load them with the pack's "LoRA Load Only" node (or any node that emits a LORA object without applying it).
• alpha_a / alpha_b - per-LoRA strength in the merge, from -10 to 10. 1.0 each is a straight sum; you can under- or over-weight one side.
• target_rank - default -1 means "auto": use min(rankA, rankB). You can force a specific rank if you want a smaller output.

One output: merged_lora, ready for Merge LoRA To Model, or Save LoRA to write it back to disk.

Installing it

It's part of ComfyUI-YogurtNodes (yogurt7771's ~150-node pack). Install with ComfyUI Manager (search "ComfyUI-YogurtNodes") or:

cd ComfyUI/custom_nodes
git clone https://github.com/yogurt7771/ComfyUI-YogurtNodes.git
cd ComfyUI-YogurtNodes
pip install -r requirements.txt

Restart ComfyUI; it's under "Yogurt Nodes". No downloads beyond the pack itself.

Where people get burned

Rank reduction is lossy. When the auto target drops a rank-128 LoRA to rank-8, the SVD keeps the 8 strongest singular directions and throws the rest away - so the merged LoRA can be noticeably weaker than stacking the originals. If you're after maximum fidelity, prefer merging LoRAs of equal rank, or use the pack's full-rank merge node which concatenates instead of compressing. Also note only layers present in both LoRAs get summed; a layer unique to B is copied over unchanged, which is usually what you want but worth knowing. And the merge happens in-memory - nothing is written to disk until you add a Save LoRA node, so don't forget that step when you want the result as a file.