Nodes/Realtime LoRA Trainer/DIT Deep Debiaser (Z-Image Sub-Component)
ComfyUI Node

DIT Deep Debiaser (Z-Image Sub-Component)

174 controls inside Z-Image, below block level

By shootthesound·Created 8 months ago·Updated 2 months ago· 538
DIT Deep Debiaser (Z-Image Sub-Component)
  • model
  • model
  • info
presetDefault
save_modelfalse
save_modefull_model
filenameauto
cap_embeddertrue
cap_embedder_str1.00
t_embeddertrue
t_embedder_str1.00
x_embeddertrue
x_embedder_str1.00
cap_pad_tokentrue
cap_pad_token_str1.00
x_pad_tokentrue
x_pad_token_str1.00
cr0_attntrue
cr0_attn_str1.00
cr0_attn_normtrue
cr0_attn_norm_str1.00
cr0_ffntrue
cr0_ffn_str1.00
cr0_ffn_normtrue
cr0_ffn_norm_str1.00
cr1_attntrue
cr1_attn_str1.00
cr1_attn_normtrue
cr1_attn_norm_str1.00
cr1_ffntrue
cr1_ffn_str1.00
cr1_ffn_normtrue
cr1_ffn_norm_str1.00
l0_adaLNtrue
l0_adaLN_str1.00
l0_attntrue
l0_attn_str1.00
l0_attn_normtrue
l0_attn_norm_str1.00
l0_ffntrue
l0_ffn_str1.00
l0_ffn_normtrue
l0_ffn_norm_str1.00
l1_adaLNtrue
l1_adaLN_str1.00
l1_attntrue
l1_attn_str1.00
l1_attn_normtrue
l1_attn_norm_str1.00
l1_ffntrue
l1_ffn_str1.00
l1_ffn_normtrue
l1_ffn_norm_str1.00
l2_adaLNtrue
l2_adaLN_str1.00
l2_attntrue
l2_attn_str1.00
l2_attn_normtrue
l2_attn_norm_str1.00
l2_ffntrue
l2_ffn_str1.00
l2_ffn_normtrue
l2_ffn_norm_str1.00
l3_adaLNtrue
l3_adaLN_str1.00
l3_attntrue
l3_attn_str1.00
l3_attn_normtrue
l3_attn_norm_str1.00
l3_ffntrue
l3_ffn_str1.00
l3_ffn_normtrue
l3_ffn_norm_str1.00
l4_adaLNtrue
l4_adaLN_str1.00
l4_attntrue
l4_attn_str1.00
l4_attn_normtrue
l4_attn_norm_str1.00
l4_ffntrue
l4_ffn_str1.00
l4_ffn_normtrue
l4_ffn_norm_str1.00
l5_adaLNtrue
l5_adaLN_str1.00
l5_attntrue
l5_attn_str1.00
l5_attn_normtrue
l5_attn_norm_str1.00
l5_ffntrue
l5_ffn_str1.00
l5_ffn_normtrue
l5_ffn_norm_str1.00
l6_adaLNtrue
l6_adaLN_str1.00
l6_attntrue
l6_attn_str1.00
l6_attn_normtrue
l6_attn_norm_str1.00
l6_ffntrue
l6_ffn_str1.00
l6_ffn_normtrue
l6_ffn_norm_str1.00
l7_adaLNtrue
l7_adaLN_str1.00
l7_attntrue
l7_attn_str1.00
l7_attn_normtrue
l7_attn_norm_str1.00
l7_ffntrue
l7_ffn_str1.00
l7_ffn_normtrue
l7_ffn_norm_str1.00
l8_adaLNtrue
l8_adaLN_str1.00
l8_attntrue
l8_attn_str1.00
l8_attn_normtrue
l8_attn_norm_str1.00
l8_ffntrue
l8_ffn_str1.00
l8_ffn_normtrue
l8_ffn_norm_str1.00
l9_adaLNtrue
l9_adaLN_str1.00
l9_attntrue
l9_attn_str1.00
l9_attn_normtrue
l9_attn_norm_str1.00
l9_ffntrue
l9_ffn_str1.00
l9_ffn_normtrue
l9_ffn_norm_str1.00
l10_adaLNtrue
l10_adaLN_str1.00
l10_attntrue
l10_attn_str1.00
l10_attn_normtrue
l10_attn_norm_str1.00
l10_ffntrue
l10_ffn_str1.00
l10_ffn_normtrue
l10_ffn_norm_str1.00
l11_adaLNtrue
l11_adaLN_str1.00
l11_attntrue
l11_attn_str1.00
l11_attn_normtrue
l11_attn_norm_str1.00
l11_ffntrue
l11_ffn_str1.00
l11_ffn_normtrue
l11_ffn_norm_str1.00
l12_adaLNtrue
l12_adaLN_str1.00
l12_attntrue
l12_attn_str1.00
l12_attn_normtrue
l12_attn_norm_str1.00
l12_ffntrue
l12_ffn_str1.00
l12_ffn_normtrue
l12_ffn_norm_str1.00
l13_adaLNtrue
l13_adaLN_str1.00
l13_attntrue
l13_attn_str1.00
l13_attn_normtrue
l13_attn_norm_str1.00
l13_ffntrue
l13_ffn_str1.00
l13_ffn_normtrue
l13_ffn_norm_str1.00
l14_adaLNtrue
l14_adaLN_str1.00
l14_attntrue
l14_attn_str1.00
l14_attn_normtrue
l14_attn_norm_str1.00
l14_ffntrue
l14_ffn_str1.00
l14_ffn_normtrue
l14_ffn_norm_str1.00
l15_adaLNtrue
l15_adaLN_str1.00
l15_attntrue
l15_attn_str1.00
l15_attn_normtrue
l15_attn_norm_str1.00
l15_ffntrue
l15_ffn_str1.00
l15_ffn_normtrue
l15_ffn_norm_str1.00
l16_adaLNtrue
l16_adaLN_str1.00
l16_attntrue
l16_attn_str1.00
l16_attn_normtrue
l16_attn_norm_str1.00
l16_ffntrue
l16_ffn_str1.00
l16_ffn_normtrue
l16_ffn_norm_str1.00
l17_adaLNtrue
l17_adaLN_str1.00
l17_attntrue
l17_attn_str1.00
l17_attn_normtrue
l17_attn_norm_str1.00
l17_ffntrue
l17_ffn_str1.00
l17_ffn_normtrue
l17_ffn_norm_str1.00
l18_adaLNtrue
l18_adaLN_str1.00
l18_attntrue
l18_attn_str1.00
l18_attn_normtrue
l18_attn_norm_str1.00
l18_ffntrue
l18_ffn_str1.00
l18_ffn_normtrue
l18_ffn_norm_str1.00
l19_adaLNtrue
l19_adaLN_str1.00
l19_attntrue
l19_attn_str1.00
l19_attn_normtrue
l19_attn_norm_str1.00
l19_ffntrue
l19_ffn_str1.00
l19_ffn_normtrue
l19_ffn_norm_str1.00
l20_adaLNtrue
l20_adaLN_str1.00
l20_attntrue
l20_attn_str1.00
l20_attn_normtrue
l20_attn_norm_str1.00
l20_ffntrue
l20_ffn_str1.00
l20_ffn_normtrue
l20_ffn_norm_str1.00
l21_adaLNtrue
l21_adaLN_str1.00
l21_attntrue
l21_attn_str1.00
l21_attn_normtrue
l21_attn_norm_str1.00
l21_ffntrue
l21_ffn_str1.00
l21_ffn_normtrue
l21_ffn_norm_str1.00
l22_adaLNtrue
l22_adaLN_str1.00
l22_attntrue
l22_attn_str1.00
l22_attn_normtrue
l22_attn_norm_str1.00
l22_ffntrue
l22_ffn_str1.00
l22_ffn_normtrue
l22_ffn_norm_str1.00
l23_adaLNtrue
l23_adaLN_str1.00
l23_attntrue
l23_attn_str1.00
l23_attn_normtrue
l23_attn_norm_str1.00
l23_ffntrue
l23_ffn_str1.00
l23_ffn_normtrue
l23_ffn_norm_str1.00
l24_adaLNtrue
l24_adaLN_str1.00
l24_attntrue
l24_attn_str1.00
l24_attn_normtrue
l24_attn_norm_str1.00
l24_ffntrue
l24_ffn_str1.00
l24_ffn_normtrue
l24_ffn_norm_str1.00
l25_adaLNtrue
l25_adaLN_str1.00
l25_attntrue
l25_attn_str1.00
l25_attn_normtrue
l25_attn_norm_str1.00
l25_ffntrue
l25_ffn_str1.00
l25_ffn_normtrue
l25_ffn_norm_str1.00
l26_adaLNtrue
l26_adaLN_str1.00
l26_attntrue
l26_attn_str1.00
l26_attn_normtrue
l26_attn_norm_str1.00
l26_ffntrue
l26_ffn_str1.00
l26_ffn_normtrue
l26_ffn_norm_str1.00
l27_adaLNtrue
l27_adaLN_str1.00
l27_attntrue
l27_attn_str1.00
l27_attn_normtrue
l27_attn_norm_str1.00
l27_ffntrue
l27_ffn_str1.00
l27_ffn_normtrue
l27_ffn_norm_str1.00
l28_adaLNtrue
l28_adaLN_str1.00
l28_attntrue
l28_attn_str1.00
l28_attn_normtrue
l28_attn_norm_str1.00
l28_ffntrue
l28_ffn_str1.00
l28_ffn_normtrue
l28_ffn_norm_str1.00
l29_adaLNtrue
l29_adaLN_str1.00
l29_attntrue
l29_attn_str1.00
l29_attn_normtrue
l29_attn_norm_str1.00
l29_ffntrue
l29_ffn_str1.00
l29_ffn_normtrue
l29_ffn_norm_str1.00
nr0_adaLNtrue
nr0_adaLN_str1.00
nr0_attntrue
nr0_attn_str1.00
nr0_attn_normtrue
nr0_attn_norm_str1.00
nr0_ffntrue
nr0_ffn_str1.00
nr0_ffn_normtrue
nr0_ffn_norm_str1.00
nr1_adaLNtrue
nr1_adaLN_str1.00
nr1_attntrue
nr1_attn_str1.00
nr1_attn_normtrue
nr1_attn_norm_str1.00
nr1_ffntrue
nr1_ffn_str1.00
nr1_ffn_normtrue
nr1_ffn_norm_str1.00
final_layertrue
final_layer_str1.00

The pack's Z-Image Model Layer Editor lets you scale whole layers of the base model. This node goes further: inside each of those layers, attention, feed-forward, and normalization are separate, individually addressable pieces - 174 of them across the whole DiT - for when a block-level edit is too blunt an instrument.

What it is and why you'd reach for it

Z-Image's DiT is built from a context_refiner, 30 main transformer layers, and a noise_refiner, and this node splits every one of them into functional sub-components rather than treating a layer as one knob. Main layers get five: adaLN (timestep and conditioning modulation), attn (self-attention: qkv, output projection, norms), attn_norm, ffn (the SwiGLU feed-forward network), and ffn_norm. The context_refiner and noise_refiner blocks get four each (no adaLN). Add five embedders (cap_embedder, t_embedder, x_embedder, cap_pad_token, x_pad_token) and one final_layer, and you land at 174 total controls.

This is the deepest edit available in the pack for Z-Image's diffusion transformer - deeper than the block-level Model Layer Editor, and operating on the base checkpoint rather than a LoRA, so whatever you do here applies to every LoRA and every prompt afterward. The pack's own author has used this level of granularity productively in public: block-level community testing on this same pack found an anatomical feature concentrated in a narrow layer band (17-21) rather than where the trainer expected, and sub-component control is the natural next step once block-level analysis has told you roughly where to look and you need finer resolution than "the whole layer."

Inputs and outputs that matter

  • model - the Z-Image checkpoint to edit.
  • preset - 15 built-in options (Weaken ALL attn 90/85/80%, Weaken ALL ffn 90/85%, Weaken ALL adaLN 90/85%, Weaken ALL attn+ffn, Weaken ALL attn_norm+ffn_norm 90%, and more) as a starting point instead of hand-setting 174 sliders.
  • cap_embedder, t_embedder, x_embedder, cap_pad_token, x_pad_token - the 5 embedder controls.
  • cr0_attn/cr0_attn_norm/cr0_ffn/cr0_ffn_norm, cr1_* - the context_refiner's two blocks, 4 controls each.
  • l0_adaLN/l0_attn/l0_attn_norm/l0_ffn/l0_ffn_norm through l29_* - the 30 main layers, 5 controls each (150 total).
  • nr0_* / nr1_* - the noise_refiner's two blocks, 5 controls each (10 total, since these do have adaLN).
  • final_layer - the last control.
  • Every one of the 174 has its own toggle and a -5 to 5 strength slider.
  • save_model / save_mode / filename (optional) - export the modified checkpoint. save_mode gives full_model (a complete, loadable checkpoint), diff_only (just the changes, applied as a patch), or both.
  • Outputs: model (patched, feed to your sampler or a LoRA loader downstream), info (a text report of what changed).

Installing it

ComfyUI Manager: search "Realtime LoRA Trainer." Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/ShootTheSound/comfyUI-Realtime-Lora

Restart ComfyUI. No additional install needed - the Deep Debiaser suite works out of the box, same as the pack's loaders and analyzers, and Z-Image's smaller size (6B parameters, quantizable down to 4GB) means this is one of the more approachable models in the pack to experiment on without a huge VRAM budget.

Troubleshooting

Set a sub-component to 0.0 for a clean removal and got noise. Expected - the pack's own warning applies here too: 0.0 tends to produce noise or artifacts rather than "no contribution," and the recommended working range is 0.5-1.5.

Not sure where to even start among 174 controls. Run the pack's block-level Model Layer Editor or one of its LoRA analyzers first to narrow down which of the 30 main layers is actually relevant to your problem, then come here and use this node's sub-component controls only on that narrower range instead of guessing across all 174 cold.

Edited ffn on several layers hoping to fix a style leak and likeness broke instead. The feed-forward network and attention do different jobs - attention tends to carry more of the relational/positional information (what's next to what), while the feed-forward network processes each token's features independently. If likeness degraded, you may have hit a layer where ffn was actually load-bearing for identity, not just style; walk the change back to a single layer at a time before applying it broadly.

Categorymodel_patches

Inputs (353)

NameTypeDefaultDescription
modelMODELZ-Image Turbo model
presetCOMBODefault15 options: Custom, Default, Weaken ALL attn 90%, Weaken ALL attn 85%, Weaken ALL attn 80%, Weaken ALL ffn 90%, +9
save_modelBOOLEANfalseSave the modified model to disk when enabled
save_modeCOMBOfull_modelfull_model = complete safetensors (~12GB). diff_only = only changed weights (small). both = save both files.
filenameSTRINGautoFilename (no extension). 'auto' = generate from settings.
cap_embedderBOOLEANtrue
cap_embedder_strFLOAT1.00-5–5
t_embedderBOOLEANtrue
t_embedder_strFLOAT1.00-5–5
x_embedderBOOLEANtrue
x_embedder_strFLOAT1.00-5–5
cap_pad_tokenBOOLEANtrue
cap_pad_token_strFLOAT1.00-5–5
x_pad_tokenBOOLEANtrue
x_pad_token_strFLOAT1.00-5–5
cr0_attnBOOLEANtrue
cr0_attn_strFLOAT1.00-5–5
cr0_attn_normBOOLEANtrue
cr0_attn_norm_strFLOAT1.00-5–5
cr0_ffnBOOLEANtrue
cr0_ffn_strFLOAT1.00-5–5
cr0_ffn_normBOOLEANtrue
cr0_ffn_norm_strFLOAT1.00-5–5
cr1_attnBOOLEANtrue
cr1_attn_strFLOAT1.00-5–5
cr1_attn_normBOOLEANtrue
cr1_attn_norm_strFLOAT1.00-5–5
cr1_ffnBOOLEANtrue
cr1_ffn_strFLOAT1.00-5–5
cr1_ffn_normBOOLEANtrue
cr1_ffn_norm_strFLOAT1.00-5–5
l0_adaLNBOOLEANtrue
l0_adaLN_strFLOAT1.00-5–5
l0_attnBOOLEANtrue
l0_attn_strFLOAT1.00-5–5
l0_attn_normBOOLEANtrue
l0_attn_norm_strFLOAT1.00-5–5
l0_ffnBOOLEANtrue
l0_ffn_strFLOAT1.00-5–5
l0_ffn_normBOOLEANtrue
l0_ffn_norm_strFLOAT1.00-5–5
l1_adaLNBOOLEANtrue
l1_adaLN_strFLOAT1.00-5–5
l1_attnBOOLEANtrue
l1_attn_strFLOAT1.00-5–5
l1_attn_normBOOLEANtrue
l1_attn_norm_strFLOAT1.00-5–5
l1_ffnBOOLEANtrue
l1_ffn_strFLOAT1.00-5–5
l1_ffn_normBOOLEANtrue
l1_ffn_norm_strFLOAT1.00-5–5
l2_adaLNBOOLEANtrue
l2_adaLN_strFLOAT1.00-5–5
l2_attnBOOLEANtrue
l2_attn_strFLOAT1.00-5–5
l2_attn_normBOOLEANtrue
l2_attn_norm_strFLOAT1.00-5–5
l2_ffnBOOLEANtrue
l2_ffn_strFLOAT1.00-5–5
l2_ffn_normBOOLEANtrue
l2_ffn_norm_strFLOAT1.00-5–5
l3_adaLNBOOLEANtrue
l3_adaLN_strFLOAT1.00-5–5
l3_attnBOOLEANtrue
l3_attn_strFLOAT1.00-5–5
l3_attn_normBOOLEANtrue
l3_attn_norm_strFLOAT1.00-5–5
l3_ffnBOOLEANtrue
l3_ffn_strFLOAT1.00-5–5
l3_ffn_normBOOLEANtrue
l3_ffn_norm_strFLOAT1.00-5–5
l4_adaLNBOOLEANtrue
l4_adaLN_strFLOAT1.00-5–5
l4_attnBOOLEANtrue
l4_attn_strFLOAT1.00-5–5
l4_attn_normBOOLEANtrue
l4_attn_norm_strFLOAT1.00-5–5
l4_ffnBOOLEANtrue
l4_ffn_strFLOAT1.00-5–5
l4_ffn_normBOOLEANtrue
l4_ffn_norm_strFLOAT1.00-5–5
l5_adaLNBOOLEANtrue
l5_adaLN_strFLOAT1.00-5–5
l5_attnBOOLEANtrue
l5_attn_strFLOAT1.00-5–5
l5_attn_normBOOLEANtrue
l5_attn_norm_strFLOAT1.00-5–5
l5_ffnBOOLEANtrue
l5_ffn_strFLOAT1.00-5–5
l5_ffn_normBOOLEANtrue
l5_ffn_norm_strFLOAT1.00-5–5
l6_adaLNBOOLEANtrue
l6_adaLN_strFLOAT1.00-5–5
l6_attnBOOLEANtrue
l6_attn_strFLOAT1.00-5–5
l6_attn_normBOOLEANtrue
l6_attn_norm_strFLOAT1.00-5–5
l6_ffnBOOLEANtrue
l6_ffn_strFLOAT1.00-5–5
l6_ffn_normBOOLEANtrue
l6_ffn_norm_strFLOAT1.00-5–5
l7_adaLNBOOLEANtrue
l7_adaLN_strFLOAT1.00-5–5
l7_attnBOOLEANtrue
l7_attn_strFLOAT1.00-5–5
l7_attn_normBOOLEANtrue
l7_attn_norm_strFLOAT1.00-5–5
l7_ffnBOOLEANtrue
l7_ffn_strFLOAT1.00-5–5
l7_ffn_normBOOLEANtrue
l7_ffn_norm_strFLOAT1.00-5–5
l8_adaLNBOOLEANtrue
l8_adaLN_strFLOAT1.00-5–5
l8_attnBOOLEANtrue
l8_attn_strFLOAT1.00-5–5
l8_attn_normBOOLEANtrue
l8_attn_norm_strFLOAT1.00-5–5
l8_ffnBOOLEANtrue
l8_ffn_strFLOAT1.00-5–5
l8_ffn_normBOOLEANtrue
l8_ffn_norm_strFLOAT1.00-5–5
l9_adaLNBOOLEANtrue
l9_adaLN_strFLOAT1.00-5–5
l9_attnBOOLEANtrue
l9_attn_strFLOAT1.00-5–5
l9_attn_normBOOLEANtrue
l9_attn_norm_strFLOAT1.00-5–5
l9_ffnBOOLEANtrue
l9_ffn_strFLOAT1.00-5–5
l9_ffn_normBOOLEANtrue
l9_ffn_norm_strFLOAT1.00-5–5
l10_adaLNBOOLEANtrue
l10_adaLN_strFLOAT1.00-5–5
l10_attnBOOLEANtrue
l10_attn_strFLOAT1.00-5–5
l10_attn_normBOOLEANtrue
l10_attn_norm_strFLOAT1.00-5–5
l10_ffnBOOLEANtrue
l10_ffn_strFLOAT1.00-5–5
l10_ffn_normBOOLEANtrue
l10_ffn_norm_strFLOAT1.00-5–5
l11_adaLNBOOLEANtrue
l11_adaLN_strFLOAT1.00-5–5
l11_attnBOOLEANtrue
l11_attn_strFLOAT1.00-5–5
l11_attn_normBOOLEANtrue
l11_attn_norm_strFLOAT1.00-5–5
l11_ffnBOOLEANtrue
l11_ffn_strFLOAT1.00-5–5
l11_ffn_normBOOLEANtrue
l11_ffn_norm_strFLOAT1.00-5–5
l12_adaLNBOOLEANtrue
l12_adaLN_strFLOAT1.00-5–5
l12_attnBOOLEANtrue
l12_attn_strFLOAT1.00-5–5
l12_attn_normBOOLEANtrue
l12_attn_norm_strFLOAT1.00-5–5
l12_ffnBOOLEANtrue
l12_ffn_strFLOAT1.00-5–5
l12_ffn_normBOOLEANtrue
l12_ffn_norm_strFLOAT1.00-5–5
l13_adaLNBOOLEANtrue
l13_adaLN_strFLOAT1.00-5–5
l13_attnBOOLEANtrue
l13_attn_strFLOAT1.00-5–5
l13_attn_normBOOLEANtrue
l13_attn_norm_strFLOAT1.00-5–5
l13_ffnBOOLEANtrue
l13_ffn_strFLOAT1.00-5–5
l13_ffn_normBOOLEANtrue
l13_ffn_norm_strFLOAT1.00-5–5
l14_adaLNBOOLEANtrue
l14_adaLN_strFLOAT1.00-5–5
l14_attnBOOLEANtrue
l14_attn_strFLOAT1.00-5–5
l14_attn_normBOOLEANtrue
l14_attn_norm_strFLOAT1.00-5–5
l14_ffnBOOLEANtrue
l14_ffn_strFLOAT1.00-5–5
l14_ffn_normBOOLEANtrue
l14_ffn_norm_strFLOAT1.00-5–5
l15_adaLNBOOLEANtrue
l15_adaLN_strFLOAT1.00-5–5
l15_attnBOOLEANtrue
l15_attn_strFLOAT1.00-5–5
l15_attn_normBOOLEANtrue
l15_attn_norm_strFLOAT1.00-5–5
l15_ffnBOOLEANtrue
l15_ffn_strFLOAT1.00-5–5
l15_ffn_normBOOLEANtrue
l15_ffn_norm_strFLOAT1.00-5–5
l16_adaLNBOOLEANtrue
l16_adaLN_strFLOAT1.00-5–5
l16_attnBOOLEANtrue
l16_attn_strFLOAT1.00-5–5
l16_attn_normBOOLEANtrue
l16_attn_norm_strFLOAT1.00-5–5
l16_ffnBOOLEANtrue
l16_ffn_strFLOAT1.00-5–5
l16_ffn_normBOOLEANtrue
l16_ffn_norm_strFLOAT1.00-5–5
l17_adaLNBOOLEANtrue
l17_adaLN_strFLOAT1.00-5–5
l17_attnBOOLEANtrue
l17_attn_strFLOAT1.00-5–5
l17_attn_normBOOLEANtrue
l17_attn_norm_strFLOAT1.00-5–5
l17_ffnBOOLEANtrue
l17_ffn_strFLOAT1.00-5–5
l17_ffn_normBOOLEANtrue
l17_ffn_norm_strFLOAT1.00-5–5
l18_adaLNBOOLEANtrue
l18_adaLN_strFLOAT1.00-5–5
l18_attnBOOLEANtrue
l18_attn_strFLOAT1.00-5–5
l18_attn_normBOOLEANtrue
l18_attn_norm_strFLOAT1.00-5–5
l18_ffnBOOLEANtrue
l18_ffn_strFLOAT1.00-5–5
l18_ffn_normBOOLEANtrue
l18_ffn_norm_strFLOAT1.00-5–5
l19_adaLNBOOLEANtrue
l19_adaLN_strFLOAT1.00-5–5
l19_attnBOOLEANtrue
l19_attn_strFLOAT1.00-5–5
l19_attn_normBOOLEANtrue
l19_attn_norm_strFLOAT1.00-5–5
l19_ffnBOOLEANtrue
l19_ffn_strFLOAT1.00-5–5
l19_ffn_normBOOLEANtrue
l19_ffn_norm_strFLOAT1.00-5–5
l20_adaLNBOOLEANtrue
l20_adaLN_strFLOAT1.00-5–5
l20_attnBOOLEANtrue
l20_attn_strFLOAT1.00-5–5
l20_attn_normBOOLEANtrue
l20_attn_norm_strFLOAT1.00-5–5
l20_ffnBOOLEANtrue
l20_ffn_strFLOAT1.00-5–5
l20_ffn_normBOOLEANtrue
l20_ffn_norm_strFLOAT1.00-5–5
l21_adaLNBOOLEANtrue
l21_adaLN_strFLOAT1.00-5–5
l21_attnBOOLEANtrue
l21_attn_strFLOAT1.00-5–5
l21_attn_normBOOLEANtrue
l21_attn_norm_strFLOAT1.00-5–5
l21_ffnBOOLEANtrue
l21_ffn_strFLOAT1.00-5–5
l21_ffn_normBOOLEANtrue
l21_ffn_norm_strFLOAT1.00-5–5
l22_adaLNBOOLEANtrue
l22_adaLN_strFLOAT1.00-5–5
l22_attnBOOLEANtrue
l22_attn_strFLOAT1.00-5–5
l22_attn_normBOOLEANtrue
l22_attn_norm_strFLOAT1.00-5–5
l22_ffnBOOLEANtrue
l22_ffn_strFLOAT1.00-5–5
l22_ffn_normBOOLEANtrue
l22_ffn_norm_strFLOAT1.00-5–5
l23_adaLNBOOLEANtrue
l23_adaLN_strFLOAT1.00-5–5
l23_attnBOOLEANtrue
l23_attn_strFLOAT1.00-5–5
l23_attn_normBOOLEANtrue
l23_attn_norm_strFLOAT1.00-5–5
l23_ffnBOOLEANtrue
l23_ffn_strFLOAT1.00-5–5
l23_ffn_normBOOLEANtrue
l23_ffn_norm_strFLOAT1.00-5–5
l24_adaLNBOOLEANtrue
l24_adaLN_strFLOAT1.00-5–5
l24_attnBOOLEANtrue
l24_attn_strFLOAT1.00-5–5
l24_attn_normBOOLEANtrue
l24_attn_norm_strFLOAT1.00-5–5
l24_ffnBOOLEANtrue
l24_ffn_strFLOAT1.00-5–5
l24_ffn_normBOOLEANtrue
l24_ffn_norm_strFLOAT1.00-5–5
l25_adaLNBOOLEANtrue
l25_adaLN_strFLOAT1.00-5–5
l25_attnBOOLEANtrue
l25_attn_strFLOAT1.00-5–5
l25_attn_normBOOLEANtrue
l25_attn_norm_strFLOAT1.00-5–5
l25_ffnBOOLEANtrue
l25_ffn_strFLOAT1.00-5–5
l25_ffn_normBOOLEANtrue
l25_ffn_norm_strFLOAT1.00-5–5
l26_adaLNBOOLEANtrue
l26_adaLN_strFLOAT1.00-5–5
l26_attnBOOLEANtrue
l26_attn_strFLOAT1.00-5–5
l26_attn_normBOOLEANtrue
l26_attn_norm_strFLOAT1.00-5–5
l26_ffnBOOLEANtrue
l26_ffn_strFLOAT1.00-5–5
l26_ffn_normBOOLEANtrue
l26_ffn_norm_strFLOAT1.00-5–5
l27_adaLNBOOLEANtrue
l27_adaLN_strFLOAT1.00-5–5
l27_attnBOOLEANtrue
l27_attn_strFLOAT1.00-5–5
l27_attn_normBOOLEANtrue
l27_attn_norm_strFLOAT1.00-5–5
l27_ffnBOOLEANtrue
l27_ffn_strFLOAT1.00-5–5
l27_ffn_normBOOLEANtrue
l27_ffn_norm_strFLOAT1.00-5–5
l28_adaLNBOOLEANtrue
l28_adaLN_strFLOAT1.00-5–5
l28_attnBOOLEANtrue
l28_attn_strFLOAT1.00-5–5
l28_attn_normBOOLEANtrue
l28_attn_norm_strFLOAT1.00-5–5
l28_ffnBOOLEANtrue
l28_ffn_strFLOAT1.00-5–5
l28_ffn_normBOOLEANtrue
l28_ffn_norm_strFLOAT1.00-5–5
l29_adaLNBOOLEANtrue
l29_adaLN_strFLOAT1.00-5–5
l29_attnBOOLEANtrue
l29_attn_strFLOAT1.00-5–5
l29_attn_normBOOLEANtrue
l29_attn_norm_strFLOAT1.00-5–5
l29_ffnBOOLEANtrue
l29_ffn_strFLOAT1.00-5–5
l29_ffn_normBOOLEANtrue
l29_ffn_norm_strFLOAT1.00-5–5
nr0_adaLNBOOLEANtrue
nr0_adaLN_strFLOAT1.00-5–5
nr0_attnBOOLEANtrue
nr0_attn_strFLOAT1.00-5–5
nr0_attn_normBOOLEANtrue
nr0_attn_norm_strFLOAT1.00-5–5
nr0_ffnBOOLEANtrue
nr0_ffn_strFLOAT1.00-5–5
nr0_ffn_normBOOLEANtrue
nr0_ffn_norm_strFLOAT1.00-5–5
nr1_adaLNBOOLEANtrue
nr1_adaLN_strFLOAT1.00-5–5
nr1_attnBOOLEANtrue
nr1_attn_strFLOAT1.00-5–5
nr1_attn_normBOOLEANtrue
nr1_attn_norm_strFLOAT1.00-5–5
nr1_ffnBOOLEANtrue
nr1_ffn_strFLOAT1.00-5–5
nr1_ffn_normBOOLEANtrue
nr1_ffn_norm_strFLOAT1.00-5–5
final_layerBOOLEANtrue
final_layer_strFLOAT1.00-5–5

Outputs (2)

NameTypeDescription
modelMODELModel with per-sub-component modifications (LoRA-safe)
infoSTRINGText summary of modifications