Nodes/ComfyUI-Inference-Core-Nodes/Inference_Core_ModelMergeSDXLTransformers
ComfyUI Node

Inference_Core_ModelMergeSDXLTransformers

Block-Level SDXL Merging, Every Slider Exposed

By LykosAI·Created 2 years ago·Updated about a year ago· 33
Inference_Core_ModelMergeSDXLTransformers
  • model1
  • model2
  • MODEL
time_embed.1.00
label_emb.1.00
input_blocks.0.0.1.00
input_blocks.1.0.1.00
input_blocks.2.0.1.00
input_blocks.3.0.1.00
input_blocks.4.0.1.00
input_blocks.4.1.1.00
input_blocks.4.1.transformer_blocks.0.1.00
input_blocks.4.1.transformer_blocks.1.1.00
input_blocks.5.0.1.00
input_blocks.5.1.1.00
input_blocks.5.1.transformer_blocks.0.1.00
input_blocks.5.1.transformer_blocks.1.1.00
input_blocks.6.0.1.00
input_blocks.7.0.1.00
input_blocks.7.1.1.00
input_blocks.7.1.transformer_blocks.0.1.00
input_blocks.7.1.transformer_blocks.1.1.00
input_blocks.7.1.transformer_blocks.2.1.00
input_blocks.7.1.transformer_blocks.3.1.00
input_blocks.7.1.transformer_blocks.4.1.00
input_blocks.7.1.transformer_blocks.5.1.00
input_blocks.7.1.transformer_blocks.6.1.00
input_blocks.7.1.transformer_blocks.7.1.00
input_blocks.7.1.transformer_blocks.8.1.00
input_blocks.7.1.transformer_blocks.9.1.00
input_blocks.8.0.1.00
input_blocks.8.1.1.00
input_blocks.8.1.transformer_blocks.0.1.00
input_blocks.8.1.transformer_blocks.1.1.00
input_blocks.8.1.transformer_blocks.2.1.00
input_blocks.8.1.transformer_blocks.3.1.00
input_blocks.8.1.transformer_blocks.4.1.00
input_blocks.8.1.transformer_blocks.5.1.00
input_blocks.8.1.transformer_blocks.6.1.00
input_blocks.8.1.transformer_blocks.7.1.00
input_blocks.8.1.transformer_blocks.8.1.00
input_blocks.8.1.transformer_blocks.9.1.00
middle_block.0.1.00
middle_block.1.1.00
middle_block.1.transformer_blocks.0.1.00
middle_block.1.transformer_blocks.1.1.00
middle_block.1.transformer_blocks.2.1.00
middle_block.1.transformer_blocks.3.1.00
middle_block.1.transformer_blocks.4.1.00
middle_block.1.transformer_blocks.5.1.00
middle_block.1.transformer_blocks.6.1.00
middle_block.1.transformer_blocks.7.1.00
middle_block.1.transformer_blocks.8.1.00
middle_block.1.transformer_blocks.9.1.00
middle_block.2.1.00
output_blocks.0.0.1.00
output_blocks.0.1.1.00
output_blocks.0.1.transformer_blocks.0.1.00
output_blocks.0.1.transformer_blocks.1.1.00
output_blocks.0.1.transformer_blocks.2.1.00
output_blocks.0.1.transformer_blocks.3.1.00
output_blocks.0.1.transformer_blocks.4.1.00
output_blocks.0.1.transformer_blocks.5.1.00
output_blocks.0.1.transformer_blocks.6.1.00
output_blocks.0.1.transformer_blocks.7.1.00
output_blocks.0.1.transformer_blocks.8.1.00
output_blocks.0.1.transformer_blocks.9.1.00
output_blocks.1.0.1.00
output_blocks.1.1.1.00
output_blocks.1.1.transformer_blocks.0.1.00
output_blocks.1.1.transformer_blocks.1.1.00
output_blocks.1.1.transformer_blocks.2.1.00
output_blocks.1.1.transformer_blocks.3.1.00
output_blocks.1.1.transformer_blocks.4.1.00
output_blocks.1.1.transformer_blocks.5.1.00
output_blocks.1.1.transformer_blocks.6.1.00
output_blocks.1.1.transformer_blocks.7.1.00
output_blocks.1.1.transformer_blocks.8.1.00
output_blocks.1.1.transformer_blocks.9.1.00
output_blocks.2.0.1.00
output_blocks.2.1.1.00
output_blocks.2.1.transformer_blocks.0.1.00
output_blocks.2.1.transformer_blocks.1.1.00
output_blocks.2.1.transformer_blocks.2.1.00
output_blocks.2.1.transformer_blocks.3.1.00
output_blocks.2.1.transformer_blocks.4.1.00
output_blocks.2.1.transformer_blocks.5.1.00
output_blocks.2.1.transformer_blocks.6.1.00
output_blocks.2.1.transformer_blocks.7.1.00
output_blocks.2.1.transformer_blocks.8.1.00
output_blocks.2.1.transformer_blocks.9.1.00
output_blocks.3.0.1.00
output_blocks.3.1.1.00
output_blocks.3.1.transformer_blocks.0.1.00
output_blocks.3.1.transformer_blocks.1.1.00
output_blocks.4.0.1.00
output_blocks.4.1.1.00
output_blocks.4.1.transformer_blocks.0.1.00
output_blocks.4.1.transformer_blocks.1.1.00
output_blocks.5.0.1.00
output_blocks.5.1.1.00
output_blocks.5.1.transformer_blocks.0.1.00
output_blocks.5.1.transformer_blocks.1.1.00
output_blocks.6.0.1.00
output_blocks.7.0.1.00
output_blocks.8.0.1.00
out.1.00

Open this node in ComfyUI for the first time and the reaction is usually the same: what is this wall of sliders. It's a legitimate technique though - block-weighted model merging, taken to its most granular extreme for SDXL's UNet. Instead of one global "how much of model A vs model B" slider, you get a separate 0–1 weight for every input block, every output block, the middle block, the time and label embeddings, and - the part that makes this the "Transformers" variant specifically - the cross-attention transformer sub-blocks inside those blocks get their own independent sliders too, down to individual transformer_blocks indices in the deeper layers.

That last bit is the whole reason this node exists instead of a plainer SDXL merge node. SDXL's UNet blocks aren't monolithic - several of them contain stacked transformer sub-layers doing the actual cross-attention work with your text conditioning. A coarser merge node blends the whole block as one unit; this one lets you blend the convolutional part of a block one way and its attention sub-layers a different way entirely. That's genuinely useful if you want to graft, say, one model's fine-detail output blocks onto another model's overall composition and prompt-following behavior, without touching everything in between uniformly.

Each slider works the same way: 1.0 is pure model1 for that block, 0.0 is pure model2, and anything between linearly interpolates. Nobody hand-tunes all ninety-something of these by trial and error in the node graph itself - in practice this gets driven by a script or preset that programmatically sets ranges (all input blocks toward model1, all output blocks toward model2, say), or by zeroing out a narrow slice to graft one specific capability across. Going in and randomly nudging individual sliders without a plan is a good way to burn an afternoon for a barely-different checkpoint.

Inputs and outputs: model1 and model2 (both MODEL) are the two checkpoints being merged - and they need to actually share the SDXL UNet architecture. Merging two genuinely different SDXL finetunes works fine; trying to merge an SDXL model with an SD 1.5 model or a DiT-based checkpoint (Flux, Qwen-Image) will not - the block structure doesn't line up, so this is not a cross-architecture tool. Every other required input is one of those per-block FLOAT weights, default 1.0, step 0.01. The output is a single merged MODEL, ready to wire straight into a KSampler like any other loaded checkpoint.

Model merging in general is one of the things ComfyUI's graph-based design makes possible that a form-based UI simply can't represent - it's called out as a canonical example of the kind of operation that only makes sense once your pipeline is a graph rather than a fixed set of buttons. This node is that idea taken about as far as it goes for SDXL specifically.

Installing it: part of LykosAI's ComfyUI-Inference-Core-Nodes pack - LykosAI also makes Stability Matrix, the meta-installer a lot of people use to manage multiple local SD UIs, so the install here tends to be more carefully put together than a typical solo node repo. Search "ComfyUI-Inference-Core-Nodes" in ComfyUI Manager, or manually: cd ComfyUI/custom_nodes && git clone https://github.com/LykosAI/ComfyUI-Inference-Core-Nodes, then run install.py against ComfyUI's own Python environment, or pip install -e .[cuda] / .[cuda12] / .[rocm] / .[directml] / .[cpu] to match your hardware, or a plain pip install -e . (this node doesn't need the onnxruntime acceleration extras - it's pure tensor arithmetic). Restart ComfyUI afterward.

Troubleshooting: the pack-wide install issue people actually hit is ModuleNotFoundError: No module named 'inference_core_nodes', meaning the git clone landed but the Python package step didn't finish - usually install.py ran against the wrong interpreter on a portable ComfyUI build. Specific to this node: both models need to be loaded in VRAM at the same time during the merge, which is a real temporary spike above what either model alone needs - if you're VRAM-constrained, that moment of the graph is where you'll feel it. And if the merge errors out or produces obvious garbage rather than a subtly-different model, the first thing to check is architecture mismatch - confirm both inputs really are SDXL checkpoints before assuming the node is broken.

Categoryadvanced/model_merging

Inputs (106)

NameTypeDefaultDescription
model1MODEL
model2MODEL
time_embed.FLOAT1.000–1
label_emb.FLOAT1.000–1
input_blocks.0.0.FLOAT1.000–1
input_blocks.1.0.FLOAT1.000–1
input_blocks.2.0.FLOAT1.000–1
input_blocks.3.0.FLOAT1.000–1
input_blocks.4.0.FLOAT1.000–1
input_blocks.4.1.FLOAT1.000–1
input_blocks.4.1.transformer_blocks.0.FLOAT1.000–1
input_blocks.4.1.transformer_blocks.1.FLOAT1.000–1
input_blocks.5.0.FLOAT1.000–1
input_blocks.5.1.FLOAT1.000–1
input_blocks.5.1.transformer_blocks.0.FLOAT1.000–1
input_blocks.5.1.transformer_blocks.1.FLOAT1.000–1
input_blocks.6.0.FLOAT1.000–1
input_blocks.7.0.FLOAT1.000–1
input_blocks.7.1.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.0.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.1.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.2.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.3.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.4.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.5.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.6.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.7.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.8.FLOAT1.000–1
input_blocks.7.1.transformer_blocks.9.FLOAT1.000–1
input_blocks.8.0.FLOAT1.000–1
input_blocks.8.1.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.0.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.1.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.2.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.3.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.4.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.5.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.6.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.7.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.8.FLOAT1.000–1
input_blocks.8.1.transformer_blocks.9.FLOAT1.000–1
middle_block.0.FLOAT1.000–1
middle_block.1.FLOAT1.000–1
middle_block.1.transformer_blocks.0.FLOAT1.000–1
middle_block.1.transformer_blocks.1.FLOAT1.000–1
middle_block.1.transformer_blocks.2.FLOAT1.000–1
middle_block.1.transformer_blocks.3.FLOAT1.000–1
middle_block.1.transformer_blocks.4.FLOAT1.000–1
middle_block.1.transformer_blocks.5.FLOAT1.000–1
middle_block.1.transformer_blocks.6.FLOAT1.000–1
middle_block.1.transformer_blocks.7.FLOAT1.000–1
middle_block.1.transformer_blocks.8.FLOAT1.000–1
middle_block.1.transformer_blocks.9.FLOAT1.000–1
middle_block.2.FLOAT1.000–1
output_blocks.0.0.FLOAT1.000–1
output_blocks.0.1.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.0.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.1.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.2.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.3.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.4.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.5.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.6.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.7.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.8.FLOAT1.000–1
output_blocks.0.1.transformer_blocks.9.FLOAT1.000–1
output_blocks.1.0.FLOAT1.000–1
output_blocks.1.1.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.0.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.1.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.2.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.3.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.4.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.5.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.6.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.7.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.8.FLOAT1.000–1
output_blocks.1.1.transformer_blocks.9.FLOAT1.000–1
output_blocks.2.0.FLOAT1.000–1
output_blocks.2.1.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.0.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.1.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.2.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.3.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.4.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.5.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.6.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.7.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.8.FLOAT1.000–1
output_blocks.2.1.transformer_blocks.9.FLOAT1.000–1
output_blocks.3.0.FLOAT1.000–1
output_blocks.3.1.FLOAT1.000–1
output_blocks.3.1.transformer_blocks.0.FLOAT1.000–1
output_blocks.3.1.transformer_blocks.1.FLOAT1.000–1
output_blocks.4.0.FLOAT1.000–1
output_blocks.4.1.FLOAT1.000–1
output_blocks.4.1.transformer_blocks.0.FLOAT1.000–1
output_blocks.4.1.transformer_blocks.1.FLOAT1.000–1
output_blocks.5.0.FLOAT1.000–1
output_blocks.5.1.FLOAT1.000–1
output_blocks.5.1.transformer_blocks.0.FLOAT1.000–1
output_blocks.5.1.transformer_blocks.1.FLOAT1.000–1
output_blocks.6.0.FLOAT1.000–1
output_blocks.7.0.FLOAT1.000–1
output_blocks.8.0.FLOAT1.000–1
out.FLOAT1.000–1

Outputs (1)

NameTypeDescription
MODELMODEL