ComfyUI Node

PM LoRA Merger

The node that turns a pile of LoRAs into one merged LoRA

By larsupb·Created 2 years ago·Updated 29 days ago· 75
PM LoRA Merger
  • method
  • components
  • strengths
  • lora
  • merge_context
output_scale1.00
spectral_norm_scale0.00
merge_cliptrue
device
dtype
refactor_methodenergy_rSVD
offload_modelstrue

This is the heart of the LoRA Power-Merger pack, the node everything else feeds into. You know the pain it solves: you stack two or three LoRAs at 0.6 strength each and the styles fight each other - character features blur, the style blend ends up mushy. Merging is the alternative: bake them into a single LoRA so the result behaves like one file instead of three layers of interference. The community's take on this pack has basically been "tried every LoRA merge method, this one gives the most flexibility with great results," and the Merger node is where that flexibility actually lands.

How it works

You don't feed raw LoRA files in here. The pack's pipeline is stack → decompose → method → merge, and this node sits at the end of it. It takes components (the per-layer (up, down, alpha) tensors from PM LoRA Stack Decompose) and strengths (the per-LoRA weights), then applies whichever method you chose.

Where the "Mergekit" in the name gets interesting: it's half true. The interpolation methods (SLERP, NuSLERP, NearSwap, SCE, Karcher) genuinely run Arcee AI's mergekit library. The task-arithmetic family (TIES, DARE, DELLA, Breadcrumbs, Task Arithmetic) has run on the pack's own delta-space implementation since 2.4.0 - it merges full dense deltas and refactors them back into a LoRA with randomized SVD. You don't need to care which code path runs; you just shouldn't expect the mergekit pip package to be the whole story.

Internally it processes layers in parallel (thread pool), validates every tensor shape before touching it, and applies strength_model to UNet layers and strength_clip to CLIP layers separately.

The inputs that matter

  • method - the MergeMethod config from whichever method node you picked (PM TIES, PM DARE, PM SLERP...). This is how you choose the algorithm.
  • components and strengths - straight from the decompose node. Wire them as-is.
  • lambda_ - a global scale on the final merged result (0–1, default 1). Dial it down if the merge comes out too hot.
  • spectral_norm_scale - the hidden gem. Set to 0 it's off; above 0 it clips any layer whose largest singular value exceeds the target, so one dominant layer can't swallow the merge. Start at 1.0 if you're getting unstable or over-strong merges.
  • merge_clip - whether to merge the text-encoder layers too. Leave on unless you're only after image-side effects.
  • device and dtype - where to compute. float16 on CUDA is fine for most SDXL work; float32 is the safe choice when results look wrong.

Two outputs: lora (the merged LoRABundle, wire it into PM LoRA Apply or PM LoRA Save) and merge_context (a reusable config that lets the Parameter Sweep Sampler re-run the merge with different settings without rebuilding the graph).

Installing

ComfyUI Manager is the easy route - search "LoRA Power-Merger" and install. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/larsupb/LoRA-Merger-ComfyUI
cd LoRA-Merger-ComfyUI
pip install -r requirements.txt

Then restart ComfyUI. The requirements are PyTorch, lxml, and mergekit installed from the Arcee git repo. Two things to know: mergekit drags a pile of its own deps into your ComfyUI Python environment, and custom nodes share one environment - if something else breaks after the install, that's the usual suspect.

Common issues

Merging LoRAs of different ranks fails unless the decompose node reconciles them first - that's what PM LoRA Stack Decompose's decomposition_method handles, so use rSVD there rather than none. And if your merge barely changes the image, check lambda_ isn't below 1 and spectral_norm_scale isn't set absurdly low. Early versions of the pack had a few rough edges (one famous "Boolean value of Tensor" error from the launch thread), but by 2.4.0 the validation is solid - errors come back structured instead of as silent garbage.

CategoryLoRA PowerMerge

Inputs (10)

NameTypeDefaultDescription
methodMergeMethod
componentsLoRATensorsThe decomposed components of the LoRAs to be merged.
strengthsLoRAWeightsThe weights of the LoRAs to be merged.
output_scaleFLOAT1.000–1Global scale (0-1) applied to the final merged result.
spectral_norm_scaleFLOAT0.000–100Spectral norm regularization. 0=disabled. >0=scale max Lipschitz constant to this value. Prevents any single layer from dominating due to large weight magnitudes. Common values: 0.5-2.0 (conservative), 1.0 (neutral), 2.0-5.0 (stronger effects).
merge_clipBOOLEANtrueWhether to merge CLIP (text encoder) layers. If False, CLIP layers are excluded from the output and LoRA Apply will not modify the CLIP model.
deviceCOMBO2 options: cuda, cpu
dtypeCOMBO3 options: float16, bfloat16, float32
refactor_methodCOMBOenergy_rSVDHow the merged dense delta is decomposed back into a LoRA (applies to the GTA family: linear, task_arithmetic, ties, dare, della, breadcrumbs). 'energy_rSVD': randomized SVD with dynamic rank chosen to retain 99% of the delta's energy (recommended). 'rSVD': randomized SVD at the fixed max input rank. Both avoid the heavy/fragile full-SVD path.
offload_modelsBOOLEANtrueEvict resident models (DIT/CLIP/VAE) from VRAM before merging, to avoid OOM on low-VRAM cards. ComfyUI reloads them automatically when the sampler runs afterward. Only acts when device=cuda; no effect on cpu merges.

Outputs (2)

NameTypeDescription
loraLoRABundle
merge_contextMergeContext