Lora Add

Merging LoRAs is a real and quietly popular pastime - search the subs and you'll find people combining style LoRAs with character LoRAs and bickering about the results for fifty comments. The catch everyone hits is ranks. If LoRA A was trained at rank 32 and LoRA B at rank 128, you can't just add their matrices together; the shapes don't line up. Lora Add is the node that squares that circle: it merges two LoRAs of different ranks using SVD-based rank alignment, and hands you one combined LoRA file.

The mechanism is more interesting than the average custom node, so it's worth understanding before you trust it. For each matching layer, it does roughly this:

1. Absorbs the alpha into the down matrix. Kohya-style LoRAs store an alpha that scales the merge at apply time (effective contribution is alpha/rank). By folding that scaling into the weights first, both LoRAs end up on comparable footing.
2. Reconstructs the layer's full update matrix - that's lora_up @ lora_down, the whole low-rank patch blown back up to full size.
3. Runs SVD on it and truncates to a target rank - default -1, which means "use the smaller of the two LoRAs' ranks." This re-factors the layer into a new up/down pair at a consistent rank.
4. Fuses with your weights: the new layer is alpha_a * A + alpha_b * B.

The inputs that matter: loraA and loraB (from Load LoRA Only), the two weights alpha_a / alpha_b (default 1.0, and negative values let you subtract one LoRA's influence), and target_rank (leave at -1 unless you specifically want the merged LoRA pinned to a higher or lower rank than the minimum). Output is a single merged_lora you can feed into Merge LoRA to Model or Save LoRA.

Now the honest part. This node is clever, but it's built for one specific flavor of LoRA: kohya-style naming with lora_down.weight / lora_up.weight keys. Only those keys go through the SVD path. Diffusers-style lora_A / lora_B keys and LyCORIS-style LoRAs don't get merged at all - the code just keeps LoRA A's version of those layers and tacks on any extras from B. If your files aren't kohya-named, you'll get a "merged" output that's mostly just LoRA A, and nothing will tell you. Check your keys with LoRA Stat Viewer before you build a whole workflow around this.

A couple of practical tips. Merging two LoRAs at the same rank? Reach for Lora Simple Add instead - it's cheaper and does a plain weighted addition without the SVD overhead. And merging is not a free lunch: you're lossy-compressing one of the LoRAs down to the other's rank, and the result is genuinely different from stacking both at runtime. Keep a copy of the originals, compare outputs at a fixed seed, and don't expect the merged file to be strictly "better" - expect it to be one file instead of two, which is often the whole point when you're shipping a workflow or a model page.

Installation is the usual no-drama story for this pack - no extra packages, no model downloads. Via ComfyUI Manager, search for Comfyui-LoraUtils; or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/lrzjason/Comfyui-LoraUtils

Restart ComfyUI and everything lands under LoraUtils. It's a small pack from lrzjason (a.k.a. xiaozhijason on Civitai), so expect hobbyist-level polish: the SVD math is genuinely there, but the edge cases - non-kohya naming chief among them - are yours to discover.