Lodestone Merge LoRAs (2)

Stacking two LoRAs in ComfyUI works right up until it doesn't. Two different character LoRAs at 0.6 each bleed into each other, a style LoRA and a detail LoRA that both touch the same blocks over-fire the weights they share, and every render costs you two LoRA passes. This node takes the other road: it expands two LoRAs into their real per-layer weight changes, merges those, and saves one new LoRA file that does the job of both.

It ships in the "Model Utility Toolkit" pack (silveroxides/ComfyUI-ModelUtils), a collection of offline model surgery tools - MetaKeys, PruneKeys, checkpoint mergers, LoRA extraction. The Lodestone family is its straightforward full-delta merger, and this two-input node is the one you'll actually reach for most days.

Why merge deltas instead of factors?

A LoRA stores a low-rank pair of matrices (down/up) that approximates a full weight change: the real effect is roughly up @ down. Most "LoRA merging" averages those compressed factors, which is like averaging two lossy JPEGs and hoping the artifacts cancel. Lodestone does the opposite - it materializes each LoRA's actual per-layer weight delta, one layer at a time, and combines those dense tensors. Merging two 128-rank LoRAs by their real effects handles the overlap honestly, and what you get out is a single full-difference .diff LoRA with alpha already normalized out of the picture.

There's no free lunch: the math is a dense matrix product per layer, so it's heavier than a factor blend, and the output is a full-size dense LoRA rather than a tiny rank-128 file. That's the point - you bake once, then the single file rides through the pack's own LoRA tooling (which reads these canonical full-difference files natively), or gets stamped into a checkpoint via LoRAMergeToModel.

What you actually set

• lora_1 / lora_2 - dropdowns of everything in your models/loras folder. lora_1 is the anchor: it supplies the metadata and defines the output layer set.
• calc_mode (default sum) - how the expanded deltas combine per layer. sum adds them; mean divides by the contributor count. The interesting ones are the Frobenius-norm-aware modes: slotnorm scales each input's delta to the median magnitude before averaging, normmatch adds the deltas then rescales the result back to a typical input's magnitude, and slotnorm-normmatch does both. If one LoRA is much "louder" than the other, plain sum double-counts its effect - that's the case the norm modes exist for. Start with sum, and if the merged result is over- or under-strong, that's your dial.
• mismatch_mode (default skip) - what happens when lora_1 has a layer the other file lacks: skip keeps LoRA 1's layer untouched, zeros treats the missing input as a zero update, error aborts rather than guess.
• output_filename - the name, without extension, of the .safetensors written to your loras folder.
• save_dtype (default bf16) and process_device (default cuda) - precision and where the per-layer math runs. If a layer blows past your VRAM, it's retried on CPU automatically.

Less commonly: exclude_patterns copies matching layers straight from LoRA 1 instead of merging them (regex per line unless you flip glob_patterns), and discard_patterns drops matching layers from the output entirely.

The only output is output_filename - this is a terminal node. Its job is producing a file, not passing a model downstream.

How it's built, and what can bite you

The node streams one logical layer at a time (via the pack's unified-efficient-loader dependency) and writes each completed delta incrementally, so memory stays near a single layer's size regardless of LoRA count. The destination file is only replaced once the whole merge finishes - a crash mid-run won't leave you a half-written LoRA. The metadata on the file records merge_method: lodestone:<calc_mode>, which is handy later when you've forgotten what you mixed.

The honest gotchas:

• It expects canonical LoRA factor pairs. DoRA magnitude vectors (dora_scale), low-bit/quantized factors, and exotic layer shapes make it raise rather than produce garbage - feed it plain LoRAs.
• Both LoRAs need matching layer keys. Layers present in only one file are exactly what mismatch_mode is for.
• Same-name runs overwrite the previous output file. Give merges distinct output_filenames unless you mean to replace them.

Installing it

ComfyUI Manager is the easy path - search for Model Utility Toolkit (the pack's display name) and install. Manually, from a terminal next to ComfyUI:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

Then restart ComfyUI. No API key, no model downloads - these nodes only read LoRAs you already own, and Manager or the pack's requirements.txt pulls in its streaming dependency (unifiedefficientloader). Find the node under ModelUtils → LoRA → Merge → Lodestone in the picker, or just search "Lodestone". You'll see its 2, 3, and Multi variants; this two-input one is the same machinery with fewer slots to fill. If you need a third input, grab Lodestone Merge LoRAs (3) - or the Multi node, which handles up to eight equal-strength LoRAs.