PM DARE

DARE (Drop And REscale) is the merge method built on a counterintuitive bet: most of a LoRA's weights are noise, so you can randomly drop most of them and the merge still works - as long as you rescale what's left. It comes from the "Language Models are Super Mario" paper, and it's been a staple of model merging for years. This node brings it to LoRA merging with a couple of modern tweaks.

When do you use it? When you're merging several LoRAs and plain averaging makes them interfere - one LoRA's changes fight another's. DARE prunes each task vector down to a sparse set before combining, which is exactly how you reduce that interference. It needs two or more LoRAs, plus it plays well with the task-vector framing (difference from a base model).

How it works

Each LoRA is turned into a task vector (its changes relative to the base). DARE randomly prunes a fraction of each vector - keep density of the weights, drop the rest - then rescales the survivors so the overall magnitude stays roughly the same, and combines them. The ties toggle is the classic upgrade: with TIES sign consensus on, instead of just summing, only weights that agree on sign across models get through, which cuts conflicting changes even harder. In mergekit terms that's dare_linear vs dare_ties.

The inputs that matter

• density - the fraction of weights retained after pruning (default 0.85). Higher keeps more signal but more interference; lower is sparser. 0.5–0.9 is the sane range; below that the merge starts to go quiet.
• ties - flip it on for sign consensus. Start with it on for multiple LoRAs; off is closer to pure DARE.
• rescale_norm - how the survivors get rescaled: l1 (preserve magnitude sum), l2 (preserve Euclidean norm), linf (cap max value), none (no rescaling - weaker result). default auto-picks.
• normalize - whether contributing weights are normalized (default true).

Output is a MergeMethod config for PM LoRA Merger. One structural note: the "(Mergekit)" in the name is a holdover - in current versions the task-arithmetic family (TIES, DARE, DELLA, Breadcrumbs) runs on the pack's own delta-space implementation, not the mergekit pip package. Doesn't change how you use it.

Installing

Ships in the LoRA Power-Merger pack. ComfyUI Manager (search "LoRA Power-Merger") or:

cd ComfyUI/custom_nodes
git clone https://github.com/larsupb/LoRA-Merger-ComfyUI
cd LoRA-Merger-ComfyUI
pip install -r requirements.txt

Restart ComfyUI. Dependencies: PyTorch, lxml, mergekit.

Common issues

The usual failure is expecting a strong result at aggressive sparsity - crank density to 0.3 and you'll get a whisper of a merge, not a bold one. Rescaling exists precisely to prevent that, so keep rescale_norm at default or l1 rather than none unless you're testing. And if your LoRAs have mismatched ranks, run them through PM LoRA Stack Decompose with rSVD first - DARE combines element-wise, so shapes must line up. If you want to find the best density without guessing, wire this method into the Parameter Sweep Sampler and sweep density from 0.5 to 0.95.