Nodes/Model Utility Toolkit/Delta CWB LoRA Multi-Merge
ComfyUI Node

Delta CWB LoRA Multi-Merge

2 to 8 LoRAs, merged by consensus instead of by average

By silveroxides·Created about a year ago·Updated a day ago· 16
Delta CWB LoRA Multi-Merge
  • cwb_config
  • output_filename
  • cwb_report
lora_count2
lora_1
lora_2
lora_3None
lora_4None
lora_5None
lora_6None
lora_7None
lora_8None
cwb_presetbalanced_mean
mismatch_modeskip
output_filenamedelta_cwb_merged_multi_lora
save_dtypebf16
output_representationfull_difference
output_rank384
process_devicecuda
exclude_patterns
discard_patterns
glob_patternsfalse
force_clear_cachetrue

A plain average of five LoRAs is how you end up with a model that looks like none of them. Delta CWB LoRA Multi-Merge is the maxed-out version of the pack's consensus merge: it takes anywhere from 2 to 8 equal-prior LoRAs, expands each into full per-layer weight changes, and merges the rows with Consensus-Weighted Blending - so a row only keeps influence if it agrees with the group consensus. The further you get from two inputs, the more that "only what agrees survives" property matters: with several LoRAs in the mix, CWB's mean-or-median consensus plus similarity weighting is closer to taking a vote than taking an average.

Like its two- and three-input siblings (Delta CWB Merge LoRAs (2) / (3)), this is a file-making node: it writes a merged .safetensors into your loras folder rather than patching anything live. LoRA 1 anchors the output layer set and metadata; every other input is an equal contributor whose actual influence is decided by the agreement math, row by row.

The inputs that matter

  • lora_count (default 2, up to 8) - must match the number of slots you actually filled. Count says 5 but lora_6 is still None? It errors out with "unselected input(s)" rather than silently merging a partial set. The count is the first thing to check when a batch merge refuses to run.
  • lora_1 through lora_8 - lora_1 and lora_2 are mandatory; the rest default to None.
  • cwb_preset (default balanced_mean) - the six presets run from simple blends (balanced_mean, robust_medn, selective_mean) to the norm-aware, diversity-boosting ones (varied_mean_rn_softcb, diverse_medn_rn_dsc_softcb, strongdiv_medn_rn_dsc_softcb). For big multi-LoRA merges the long-name presets exist specifically to keep one dominant LoRA from flattening the rest - when you've got five inputs of very different strengths, diverse_medn_rn_dsc_softcb or strongdiv_medn_rn_dsc_softcb are worth testing first.
  • cwb_config (optional) - connect the pack's CWB Custom Configuration node and it replaces the preset entirely.
  • mismatch_mode (default skip) - with up to eight independently trained LoRAs, missing layers aren't a corner case. skip keeps LoRA 1's version, zeros treats absent inputs as zero contribution, error aborts.
  • output_representation / output_rank - full_difference saves the exact merged dense deltas; factorized_lora re-factorizes each merged layer into compact lora_A/lora_B factors up to output_rank (default 384) if you want a conventional LoRA-shaped file instead.
  • output_filename, save_dtype (default bf16), process_device (CUDA with per-layer CPU fallback), exclude_patterns/discard_patterns/glob_patterns, and force_clear_cache complete the panel.

Outputs are output_filename and cwb_report - the report lists merged vs. anchor-preserved layers, zero contributors, rejected unsupported layers, and CUDA→CPU fallbacks. With many inputs, that report is your only window into whether the merge did what you hoped; wire it to a text node and read it.

Honest expectations

This is the pack's most experimental corner - the whole Delta CWB family is marked experimental in the source, with no real community track record yet, so verify results against a plain stacked-LoRA render before you commit. It only merges canonical LoRA factor pairs; DoRA-scale and low-bit factors raise per layer. Streaming one layer at a time keeps memory proportional to a single layer rather than eight full LoRAs, and a layer that OOMs on CUDA retries on CPU. Output is atomic: a failed run leaves any existing same-named file untouched, but a successful rerun overwrites it - name your merges.

Installing it

ComfyUI Manager: search Model Utility Toolkit. By hand:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

Restart ComfyUI. No API keys, no downloads - it reads only the LoRAs already in your models/loras folder. Find it under ModelUtils → LoRA → Merge → Delta CWB or search "Delta", and grab the Multi entry (the 2 and 3 variants are fixed-slot versions of identical machinery). If you want the same batch-expand idea without the consensus weighting, the Lodestone Multi merger in the same pack is the simpler, norm-based alternative.

CategoryModelUtils/LoRA/Merge/Delta CWB

Inputs (21)

NameTypeDefaultDescription
lora_countCOMBO2Number of consecutive selected LoRAs to merge.
lora_1COMBOMetadata and layer anchor. Defines saved layers and preserved values.
lora_2COMBOSecond equal-prior LoRA contributor.
lora_3COMBONoneAdditional equal-prior contributor; required when included by LoRA Count.
lora_4COMBONoneAdditional equal-prior contributor; required when included by LoRA Count.
lora_5COMBONoneAdditional equal-prior contributor; required when included by LoRA Count.
lora_6COMBONoneAdditional equal-prior contributor; required when included by LoRA Count.
lora_7COMBONoneAdditional equal-prior contributor; required when included by LoRA Count.
lora_8COMBONoneAdditional equal-prior contributor; required when included by LoRA Count.
cwb_presetCOMBObalanced_meanControls how strongly agreement, disagreement, and row magnitude affect the saved full weight changes. A connected CWB Config overrides it completely.
mismatch_modeCOMBOskipFor a missing or incompatible anchored layer: preserve LoRA 1, include an explicit zero contributor, or abort.
output_filenameSTRINGdelta_cwb_merged_multi_loraRelative filename for the atomically saved full-difference LoRA.
save_dtypeCOMBObf16Requested saved precision. Participating FP32 inputs preserve FP32.
output_representationCOMBOfull_differenceSave exact full differences or factorize each merged layer directly into compact LoRA factors.
output_rankINT3841–16384Maximum rank used when Output Representation is factorized_lora.
process_deviceCOMBOcudaProcessing device. Only a failed CUDA layer retries on CPU.
exclude_patternsSTRINGPreserve matching canonical full-difference layers from LoRA 1.
discard_patternsSTRINGOmit matching canonical full-difference layers from the output.
glob_patternsBOOLEANfalseUse shell-style glob patterns instead of regular expressions.
force_clear_cacheBOOLEANtrueClear Python and CUDA caches before each layer to reduce retained memory.
cwb_configoptCWB_CONFIGOptional complete settings override from CWB Custom Configuration.

Outputs (2)

NameTypeDescription
output_filename*
cwb_reportSTRING