🧬 XS · DARE TIES Merge Models

Task-arithmetic / DARE / TIES merge of an SDXL stack against a reference model. Per child i, base delta is (child_i - reference). Then optionally DARE drop+rescale, optionally TIES trim (keep top ties_density fraction by magnitude). Weighted contribution is w_i * block_scale_i(k) * trimmed_delta. merge_mode: weighted_delta - sum weighted contributions (plain task arithmetic). dare_delta - DARE each child delta, then sum. ties_delta - TIES trim each child, two-pass sign election on signed sum, then sum sign-aligned contributions only. dare_ties_delta - DARE then TIES trim, then same two-pass TIES as ties_delta. Final: out = reference + delta_strength * merged_delta (same for CLIP via merge_clip). Order semantics: weighted_delta is mathematically commutative across stack order (subject to fp32 rounding). dare_delta / ties_delta / dare_ties_delta are deterministic for a fixed (stack order, seed, key, child index). stable_key_seed(seed, key, i) binds DARE masks to slot index i, so reordering entries changes masks and can change TIES sign election — order-sensitive. Optional per-child profiles multiply each child's delta per UNet block (21 samples: 9 IN + 3 MID + 9 OUT). CLIP merge ignores profiles. VAE is passed through from the reference. CLIP merging is opt-in via merge_clip. **merge_math_gpu:** when True (and CUDA is available), run delta / DARE / TIES accumulate math on the GPU. Often faster than CPU fp32 at the cost of **much higher VRAM** during merge (especially ties_delta / dare_ties_delta). Leave False if you OOM.

By x-stride·Created 3 months ago·Updated 3 months ago· 0

reference_model
reference_clip
reference_vae
model_stack

model
clip
vae
merge_report

◄merge_modedare_delta►

◄drop_rate0.10►

◄delta_strength1.00►

◄seed1234►

◄normalize_model_weightstrue►

◄merge_clipoff►

◄normalize_clip_weightstrue►

◄clip_delta_strength0.50►

◄ties_density0.50►

◄merge_math_gpufalse►

Categoryadvanced/model_merging/dare_ties

Inputs (14)

Name	Type	Default	Description
reference_model	MODEL		Base / origin model. Children are merged in as deltas relative to this. Typically the base Illustrious or SDXL checkpoint.
reference_clip	CLIP		Reference CLIP. Used as origin for CLIP delta computation, or passed straight through if merge_clip='off'.
reference_vae	VAE		Reference VAE. Always passed through unchanged; included so the node can return a complete (MODEL, CLIP, VAE) triplet.
model_stack	DARE_TIES_MODEL_STACK		Stack of child checkpoints from one or more DARE TIES Model Stack nodes.
merge_mode	COMBO	dare_delta	weighted_delta: plain weighted task arithmetic. Stable baseline; ignores drop_rate/ties_density. dare_delta: DARE drop-and-rescale per child delta, then sum. Uses drop_rate. ties_delta: TIES trim per child, then two-pass sign-aligned sum. Uses ties_density. Resolves sign conflicts between checkpoints (style vs anatomy fights, opposing directions). dare_ties_delta: DARE then TIES trim, then two-pass sign-aligned sum. Uses both. Recommended starting recipe: drop_rate=0.10, ties_density=0.50.
drop_rate	FLOAT	0.100–0.95	DARE drop probability p. Each delta element is dropped with prob p, survivors rescaled by 1/(1-p), preserving expectation. Used when merge_mode or merge_clip is dare_delta or dare_ties_delta. Ignored otherwise. Sane ranges: 0.05-0.10 conservative, 0.10-0.25 normal, 0.25-0.40 aggressive, 0.50+ experimental. Larger stacks tolerate less drop: 9 children -> start at 0.05-0.10.
delta_strength	FLOAT	1.00-2–2	Scalar multiplier on the summed model delta. Applied after DARE/TIES, before patching. 1.0 = full merge (with normalize_model_weights=True this is a true weighted average), <1.0 stays closer to reference, >1.0 extrapolates beyond children, negative values invert the merge direction. Sane ranges: 0.7-1.0 closer to base, 1.0 normal, 1.0-1.25 stronger child influence.
seed	INT	12340–18446744073709550000	Seed for the DARE Bernoulli mask. Combined per-key and per-child slot index via blake2b for deterministic, reproducible masks. Same seed + same stack order -> identical merge. Only affects modes that use DARE (dare_delta, dare_ties_delta).
normalize_model_weights	BOOLEAN	true	If True, model_weight values across the active stack entries are rescaled to sum to 1.0 (true weighted average; combined with delta_strength=1.0 this is a clean N-way blend). If False, raw weights are used as multipliers (sums can exceed 1.0; effectively boosts child influence).
merge_clip	COMBO	off	off: pass reference CLIP through unchanged (recommended default for SDXL; text encoders are sensitive to merging). weighted_delta / dare_delta / ties_delta / dare_ties_delta: same semantics as merge_mode, applied to the text encoder stack. Reuses drop_rate and ties_density from above. Test only when prompts feel too close to base or children clearly altered text behavior.
normalize_clip_weights	BOOLEAN	true	Same as normalize_model_weights but applied to clip_weight values across active stack entries. Ignored when merge_clip='off'.
clip_delta_strength	FLOAT	0.50-2–2	Scalar multiplier on the summed CLIP delta (analogue of delta_strength for the text encoder). Default 0.5 is conservative because text encoders are sensitive to merging. Sane ranges: 0.25-0.50 conservative, 0.50-0.75 normal, 0.75-1.00 aggressive. Ignored when merge_clip='off'.
ties_density	FLOAT	0.500.01–1	TIES trim density: keep the largest-magnitude ties_density fraction of each child delta (zero the rest), then run sign-aligned merge. 1.0 keeps everything (sign election only); lower values discard more of each child's small noise. Used when merge_mode or merge_clip is ties_delta or dare_ties_delta. Ignored otherwise. Sane ranges: 0.70-1.00 conservative, 0.40-0.70 normal, 0.20-0.40 aggressive.
merge_math_gpu	BOOLEAN	false	If True (and CUDA is available), run weighted delta / DARE / TIES math on GPU — often faster than CPU fp32. Uses substantially more VRAM while merging (TIES two-pass keeps large fp32 tensors hot). Disable if you hit OOM; keep False for lowest VRAM. Placed last so saved workflows match earlier widgets by index (don’t reorder above).

Outputs (4)

Name	Type	Description
model	MODEL	Merged MODEL: reference_model with summed child deltas applied as a single 'diff' patch per merged key.
clip	CLIP	Merged CLIP (cloned reference if merge_clip='off' or unchanged on the off-path).
vae	VAE	Reference VAE, passed through unchanged.
merge_report	STRING	Human-readable report: stack contents, profiles, effective weights, drop rate, TIES density, delta strengths, and per-section merged/skipped/mismatch key counts.