Delta CWB Merge LoRAs (2)

Here's the problem with merging two LoRAs by averaging: when both LoRAs agree on a weight change, the average keeps it, but when they disagree - say two character LoRAs both pushing different faces at the same layer - a plain mean quietly splits the difference and you get neither. Delta CWB Merge LoRAs (2) is a smarter take on that problem. It expands two LoRAs into full per-layer weight changes, then merges them row by row using Consensus-Weighted Blending: rows where both LoRAs point the same way get kept and boosted, rows where they fight each other get down-weighted. The result is one merged full-difference LoRA that preserves the shared intent instead of the compromise.

"CWB" (Consensus-Weighted Blending) is the pack author's own merge algorithm, and this Delta CWB family is its LoRA-specific application. Note the honest label: in the source these nodes are flagged experimental, and they've seen essentially zero community mileage so far. That's not a reason to avoid them - it's a reason to verify outputs instead of trusting a recipe you read once.

How the merge works

A LoRA's real effect per layer is a dense matrix (its up @ down product, alpha normalized). This node expands both LoRAs to those full per-layer deltas, then for each corresponding row vector it:

1. computes a consensus (mean or median) of the contributors,
2. measures each row's cosine similarity to that consensus,
3. weights rows by how much they agree - disagreement gets suppressed,
4. normalizes the weights and sums.

The preset you pick tunes how aggressively all that happens. The mechanism preserves model coordinates; it doesn't reorder or re-rank anything. Inputs are equal-prior - the agreement math decides who actually influences each row. Because the merged result is saved, not applied, what you get is a new .safetensors in your loras folder (.diff tensors, alpha normalized out) that the pack's own LoRA tooling reads natively - inspect it, or stamp it into a checkpoint with LoRAMergeToModel.

Inputs worth your attention

• cwb_preset (default balanced_mean) - the six presets span a spectrum. The three simple ones (balanced_mean, robust_medn, selective_mean) blend by mean or median with increasing discrimination; the three long-named ones (varied_mean_rn_softcb, diverse_medn_rn_dsc_softcb, strongdiv_medn_rn_dsc_softcb) add rescale-norm, similarity contrast, and a "soft comfort bandpass" that pushes toward diversity. Preset names are legible once you know the tokens: _mean/_medn = consensus type, _rn = rescale norm, _dsc = dynamic similarity contrast, _softcb = soft comfort bandpass. Start at the default.
• cwb_config (optional) - connect the pack's CWB Custom Configuration node and it completely overrides the preset, giving you every knob individually. Leave it unconnected and the preset rules.
• mismatch_mode (default skip) - missing or incompatible layers: preserve LoRA 1's layer, insert an explicit zero contributor, or abort.
• output_representation (default full_difference) - save the merged dense deltas as-is, or factorized_lora, which re-factorizes each merged layer into compact lora_A/lora_B factors up to output_rank (default 384). Choose factorization if you want a smaller, more conventional LoRA-shaped file; pick full-difference if you want the exact merged result preserved.
• output_filename, save_dtype (default bf16), process_device (default cuda, per-layer CPU retry on OOM), and exclude_patterns/discard_patterns/glob_patterns for layer-level surgery. force_clear_cache (default on) releases Python/CUDA caches between layers to cap memory.

Two outputs, which is the giveaway that this family is different from the Lodestone merges: output_filename (the created file) and cwb_report - a text summary of merged layers, anchor-preserved layers, zero contributors, rejected/unsupported layers, CUDA→CPU fallbacks, and output dtypes. Wire cwb_report to a text display node and you can see exactly what the merge decided instead of trusting it blindly.

Common issues

• Unsupported tensors are rejected per layer, not silently dropped - DoRA-scale vectors and low-bit factors make the node raise. Plain LoRA factor pairs are the target input.
• The destination file is written atomically and only replaced on full success, so a failed run won't destroy an existing LoRA of the same name - but a successful rerun will overwrite it. Name merges deliberately.
• "Expanded to full weight changes" means dense math per layer. It streams one layer at a time, but expect a bigger run and heavier VRAM use than a factor-level blend.

Installing it

ComfyUI Manager: search Model Utility Toolkit. By hand:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

Restart ComfyUI - no API keys, no model downloads, it only reads LoRAs already in models/loras. Find it under ModelUtils → LoRA → Merge → Delta CWB, or search "Delta". If you need a third or fourth contributor, the 3 and Multi variants in the same menu use identical controls; when two equal-strength LoRAs are all you have, this two-input version is where the CWB idea is easiest to evaluate.