LoRA Multi-Merge (DARE-Ties)

Merge more than two LoRAs and the naive approach - just average the weights - tends to produce mush, especially when the LoRAs overlap on the same layers (two character LoRAs both wanting to push the same attention heads in different directions is the classic case). DARE-Ties is the fix the LLM model-merging world came up with, and this node, from silveroxides' ComfyUI-ModelUtils, brings it to LoRAs directly inside ComfyUI: merge 1-8 of them into a single file, with the drop-and-resolve logic doing the work a flat average can't.

The mechanism. DARE-Ties is two ideas stacked. DARE (Drop And REscale) randomly zeroes out a fraction of each LoRA's small-magnitude weight changes before merging - most of a weight delta is often noise anyway, and dropping it (then rescaling what's left) turns out to preserve the signal better than keeping everything. TIES (the trim-elect-sign-merge idea) handles the case where multiple LoRAs disagree about which direction a shared parameter should move: it trims the smallest-magnitude changes, then resolves sign conflicts by picking whichever direction has the larger combined support, instead of letting opposing pushes cancel out into nothing. Put together, that's what the node's own description says plainly: "Drops small values and resolves sign conflicts." This isn't a fringe idea, either - it's community-verified as a real, current merging technique people actually reach for over plain weighted averaging when combining multiple LoRAs.

Inputs that matter. lora_1 through lora_8 are your source LoRAs (only lora_1 is required; the rest default to None), each with a matching weight_N (default 1.0, range -10 to 10) controlling its contribution before dropping and merging happens. lora_count tells the node how many of those eight slots are actually active. base_model (default None) is worth understanding: if your LoRAs came from different trainers or tools, their internal tensor key names can differ even when they target the same architecture - pointing base_model at the actual checkpoint they're meant for lets the node use canonical naming to line everything up correctly; leave it None only if you're confident all your LoRAs came from the same pipeline. drop_rate (default 0.1) is DARE's drop fraction, trim_quantile (default 0.2) is TIES' trim threshold. seed (default 42) matters because the dropping step is randomized - pin it if you want a reproducible merge. output_filename, save_dtype (fp16), and device (cuda) round it out. One output: output_path.

Installing it. ComfyUI Manager, search "Model Utility Toolkit" - or cd ComfyUI/custom_nodes && git clone https://github.com/silveroxides/ComfyUI-ModelUtils and restart. No extra dependencies flagged in the README; you supply the LoRAs.

Tuning without guessing. If a merge comes out weak or washed-out, drop_rate or trim_quantile set too aggressively is the usual suspect - both are trimming away information, and pushed too far they trim away the parts that actually mattered. Back them off before touching individual LoRA weights. If two LoRAs are actively fighting (same concept, conflicting execution), TIES' sign resolution should handle it better than a plain average would, but it's not magic - wildly incompatible LoRAs will still merge into something incoherent, they'll just do it slightly more gracefully than a naive sum.

Where it bites. All your LoRAs need to target the same base architecture - DARE-Ties resolves conflicts between compatible deltas, it doesn't reconcile LoRAs trained for genuinely different models. And worth knowing if you go looking further: DARE-Ties isn't the newest merging idea out there (a method called DELLA has come up more recently in community discussion), but it's solid, well-understood, and it's what this pack ships. If a flat weighted sum has been giving you mediocre multi-LoRA merges, this is the upgrade to try first.