CWB Merge LoRAs (2 Models)

Merging two LoRAs into one file is a classic move - you stop juggling two LoRA loader nodes at carefully balanced strengths and just load a single file. The catch has always been that naive blending averages the raw down/up matrices, which is wrong in ways that show up as degraded output. CWB's per-layer consensus weighting is a meaningfully better approach here, and this node applies it with LoRA structure in mind.

How it works

A LoRA layer is a down tensor and an up tensor that multiply to the effective delta. This merger treats each pair as one logical unit, so the blend happens on real layers, not individual matrices. Model A is the anchor (metadata, naming, and preservation reference); Model B is the second equal-prior contributor. For each logical layer, CWB computes a consensus (mean or median), measures each LoRA's similarity to it, and weights contributions by that similarity - so a layer where the two LoRAs agree gets averaged calmly, and a layer where they conflict gets a considered blend rather than a straight mean.

Because LoRA rows don't have a fixed semantic order, the LoRA presets default to similarity-based alignment (_sim in the name) rather than index alignment. The default preset is broad_sim_medn_rn_softcb: median consensus, similarity alignment, norm rescaling, and the soft comfort bandpass. The preset menu has gentler and harsher variants - moderate_... and conservative_... raise the alignment threshold, focused_... and strongfocus_... add dynamic similarity contrast to sharpen the differences.

Inputs that matter

• execution_mode - MERGE writes the file; DOCUMENTATION ONLY returns the CWB reference without loading.
• model_a / model_b - the two LoRAs.
• cwb_preset (default broad_sim_medn_rn_softcb) - the LoRA-specific preset registry described above.
• include_1d_diffs (default off) - LoRA files carry 1D tensors (alphas, biases) that aren't down/up pairs. Off preserves Model A's copies; on CWB-merges them in fp32. Leave off unless you know you need it.
• output_filename (default cwb_merged_2_lora) - written to models/loras.
• save_dtype / override_dtype - dtype for generated factors; fp32 inputs stay fp32 unless forced.
• mismatch_mode - skip preserves the anchor, zeros inserts zero contributions, error aborts for missing/incompatible groups.
• exclude_patterns / discard_patterns / glob_patterns - regex (or glob) lists; excludes preserve matching logical groups from the anchor, discards drop them.
• lazy_load (default on), force_clear_cache (default on), process_device - the usual UEL streaming, per-layer cache flush, and CUDA-with-CPU-fallback.

Outputs

output_filename (the written file), documentation (the CWB reference), and cwb_report (per-layer merge detail). Load the merged LoRA with a normal LoRA Loader and apply it at whatever strength you want.

What this is really for

Two LoRAs trained on the same character or style, each good at different things, and you want one file. Or the "merge my LoRA into the one I downloaded" cleanup. It also sidesteps a real workflow problem - stacked LoRAs at high combined strength is how you get that overcooked look; a single merged file behaves more like a coherent fine-tune. Run LoRAModelAnalysis first if you want to see whether the two actually overlap before committing.

Install

Part of Model Utility Toolkit (silveroxides/ComfyUI-ModelUtils). ComfyUI Manager → search "Model Utility Toolkit", or:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

Restart ComfyUI. Real dependency: unifiedefficientloader (UEL). Keep ComfyUI current - newer extension API, and the CWB nodes are author-flagged experimental.