PM Dare (Mergekit)

Concept: Similar to TIES, DARE sparsifies task vectors to reduce interference. However, DARE uses random pruning with a novel rescaling technique to better match the performance of the original models. Variants: - dare_linear: DARE pruning without the TIES sign consensus - dare_ties: DARE pruning with the TIES sign consensus Use Cases: - Robustly combining multiple fine-tuned models, often yielding better performance than TIES in some scenarios - Inputs: Requires 2 or more models. Key Parameters: - weight (per-model): Weight for each model's task vector - density (per-model): Fraction of weights to retain after random pruning - lambda (global): As in Task Arithmetic - rescale (global, for dare_linear): If true (default), applies DARE's rescaling