Nodes/Realtime LoRA Trainer/Text Encoder Deep Debiaser (Qwen3-8B / Flux 2 Klein)
ComfyUI Node

Text Encoder Deep Debiaser (Qwen3-8B / Flux 2 Klein)

182 sub-component dials for Klein's encoder

By shootthesound·Created 8 months ago·Updated 2 months ago· 538
Text Encoder Deep Debiaser (Qwen3-8B / Flux 2 Klein)
  • clip
  • clip
  • info
presetDefault
save_modelfalse
save_modefull_model
filenameauto
embed_tokenstrue
embed_tokens_str1.00
l0_input_normtrue
l0_input_norm_str1.00
l0_attntrue
l0_attn_str1.00
l0_attn_normtrue
l0_attn_norm_str1.00
l0_mlptrue
l0_mlp_str1.00
l0_post_normtrue
l0_post_norm_str1.00
l1_input_normtrue
l1_input_norm_str1.00
l1_attntrue
l1_attn_str1.00
l1_attn_normtrue
l1_attn_norm_str1.00
l1_mlptrue
l1_mlp_str1.00
l1_post_normtrue
l1_post_norm_str1.00
l2_input_normtrue
l2_input_norm_str1.00
l2_attntrue
l2_attn_str1.00
l2_attn_normtrue
l2_attn_norm_str1.00
l2_mlptrue
l2_mlp_str1.00
l2_post_normtrue
l2_post_norm_str1.00
l3_input_normtrue
l3_input_norm_str1.00
l3_attntrue
l3_attn_str1.00
l3_attn_normtrue
l3_attn_norm_str1.00
l3_mlptrue
l3_mlp_str1.00
l3_post_normtrue
l3_post_norm_str1.00
l4_input_normtrue
l4_input_norm_str1.00
l4_attntrue
l4_attn_str1.00
l4_attn_normtrue
l4_attn_norm_str1.00
l4_mlptrue
l4_mlp_str1.00
l4_post_normtrue
l4_post_norm_str1.00
l5_input_normtrue
l5_input_norm_str1.00
l5_attntrue
l5_attn_str1.00
l5_attn_normtrue
l5_attn_norm_str1.00
l5_mlptrue
l5_mlp_str1.00
l5_post_normtrue
l5_post_norm_str1.00
l6_input_normtrue
l6_input_norm_str1.00
l6_attntrue
l6_attn_str1.00
l6_attn_normtrue
l6_attn_norm_str1.00
l6_mlptrue
l6_mlp_str1.00
l6_post_normtrue
l6_post_norm_str1.00
l7_input_normtrue
l7_input_norm_str1.00
l7_attntrue
l7_attn_str1.00
l7_attn_normtrue
l7_attn_norm_str1.00
l7_mlptrue
l7_mlp_str1.00
l7_post_normtrue
l7_post_norm_str1.00
l8_input_normtrue
l8_input_norm_str1.00
l8_attntrue
l8_attn_str1.00
l8_attn_normtrue
l8_attn_norm_str1.00
l8_mlptrue
l8_mlp_str1.00
l8_post_normtrue
l8_post_norm_str1.00
l9_input_normtrue
l9_input_norm_str1.00
l9_attntrue
l9_attn_str1.00
l9_attn_normtrue
l9_attn_norm_str1.00
l9_mlptrue
l9_mlp_str1.00
l9_post_normtrue
l9_post_norm_str1.00
l10_input_normtrue
l10_input_norm_str1.00
l10_attntrue
l10_attn_str1.00
l10_attn_normtrue
l10_attn_norm_str1.00
l10_mlptrue
l10_mlp_str1.00
l10_post_normtrue
l10_post_norm_str1.00
l11_input_normtrue
l11_input_norm_str1.00
l11_attntrue
l11_attn_str1.00
l11_attn_normtrue
l11_attn_norm_str1.00
l11_mlptrue
l11_mlp_str1.00
l11_post_normtrue
l11_post_norm_str1.00
l12_input_normtrue
l12_input_norm_str1.00
l12_attntrue
l12_attn_str1.00
l12_attn_normtrue
l12_attn_norm_str1.00
l12_mlptrue
l12_mlp_str1.00
l12_post_normtrue
l12_post_norm_str1.00
l13_input_normtrue
l13_input_norm_str1.00
l13_attntrue
l13_attn_str1.00
l13_attn_normtrue
l13_attn_norm_str1.00
l13_mlptrue
l13_mlp_str1.00
l13_post_normtrue
l13_post_norm_str1.00
l14_input_normtrue
l14_input_norm_str1.00
l14_attntrue
l14_attn_str1.00
l14_attn_normtrue
l14_attn_norm_str1.00
l14_mlptrue
l14_mlp_str1.00
l14_post_normtrue
l14_post_norm_str1.00
l15_input_normtrue
l15_input_norm_str1.00
l15_attntrue
l15_attn_str1.00
l15_attn_normtrue
l15_attn_norm_str1.00
l15_mlptrue
l15_mlp_str1.00
l15_post_normtrue
l15_post_norm_str1.00
l16_input_normtrue
l16_input_norm_str1.00
l16_attntrue
l16_attn_str1.00
l16_attn_normtrue
l16_attn_norm_str1.00
l16_mlptrue
l16_mlp_str1.00
l16_post_normtrue
l16_post_norm_str1.00
l17_input_normtrue
l17_input_norm_str1.00
l17_attntrue
l17_attn_str1.00
l17_attn_normtrue
l17_attn_norm_str1.00
l17_mlptrue
l17_mlp_str1.00
l17_post_normtrue
l17_post_norm_str1.00
l18_input_normtrue
l18_input_norm_str1.00
l18_attntrue
l18_attn_str1.00
l18_attn_normtrue
l18_attn_norm_str1.00
l18_mlptrue
l18_mlp_str1.00
l18_post_normtrue
l18_post_norm_str1.00
l19_input_normtrue
l19_input_norm_str1.00
l19_attntrue
l19_attn_str1.00
l19_attn_normtrue
l19_attn_norm_str1.00
l19_mlptrue
l19_mlp_str1.00
l19_post_normtrue
l19_post_norm_str1.00
l20_input_normtrue
l20_input_norm_str1.00
l20_attntrue
l20_attn_str1.00
l20_attn_normtrue
l20_attn_norm_str1.00
l20_mlptrue
l20_mlp_str1.00
l20_post_normtrue
l20_post_norm_str1.00
l21_input_normtrue
l21_input_norm_str1.00
l21_attntrue
l21_attn_str1.00
l21_attn_normtrue
l21_attn_norm_str1.00
l21_mlptrue
l21_mlp_str1.00
l21_post_normtrue
l21_post_norm_str1.00
l22_input_normtrue
l22_input_norm_str1.00
l22_attntrue
l22_attn_str1.00
l22_attn_normtrue
l22_attn_norm_str1.00
l22_mlptrue
l22_mlp_str1.00
l22_post_normtrue
l22_post_norm_str1.00
l23_input_normtrue
l23_input_norm_str1.00
l23_attntrue
l23_attn_str1.00
l23_attn_normtrue
l23_attn_norm_str1.00
l23_mlptrue
l23_mlp_str1.00
l23_post_normtrue
l23_post_norm_str1.00
l24_input_normtrue
l24_input_norm_str1.00
l24_attntrue
l24_attn_str1.00
l24_attn_normtrue
l24_attn_norm_str1.00
l24_mlptrue
l24_mlp_str1.00
l24_post_normtrue
l24_post_norm_str1.00
l25_input_normtrue
l25_input_norm_str1.00
l25_attntrue
l25_attn_str1.00
l25_attn_normtrue
l25_attn_norm_str1.00
l25_mlptrue
l25_mlp_str1.00
l25_post_normtrue
l25_post_norm_str1.00
l26_input_normtrue
l26_input_norm_str1.00
l26_attntrue
l26_attn_str1.00
l26_attn_normtrue
l26_attn_norm_str1.00
l26_mlptrue
l26_mlp_str1.00
l26_post_normtrue
l26_post_norm_str1.00
l27_input_normtrue
l27_input_norm_str1.00
l27_attntrue
l27_attn_str1.00
l27_attn_normtrue
l27_attn_norm_str1.00
l27_mlptrue
l27_mlp_str1.00
l27_post_normtrue
l27_post_norm_str1.00
l28_input_normtrue
l28_input_norm_str1.00
l28_attntrue
l28_attn_str1.00
l28_attn_normtrue
l28_attn_norm_str1.00
l28_mlptrue
l28_mlp_str1.00
l28_post_normtrue
l28_post_norm_str1.00
l29_input_normtrue
l29_input_norm_str1.00
l29_attntrue
l29_attn_str1.00
l29_attn_normtrue
l29_attn_norm_str1.00
l29_mlptrue
l29_mlp_str1.00
l29_post_normtrue
l29_post_norm_str1.00
l30_input_normtrue
l30_input_norm_str1.00
l30_attntrue
l30_attn_str1.00
l30_attn_normtrue
l30_attn_norm_str1.00
l30_mlptrue
l30_mlp_str1.00
l30_post_normtrue
l30_post_norm_str1.00
l31_input_normtrue
l31_input_norm_str1.00
l31_attntrue
l31_attn_str1.00
l31_attn_normtrue
l31_attn_norm_str1.00
l31_mlptrue
l31_mlp_str1.00
l31_post_normtrue
l31_post_norm_str1.00
l32_input_normtrue
l32_input_norm_str1.00
l32_attntrue
l32_attn_str1.00
l32_attn_normtrue
l32_attn_norm_str1.00
l32_mlptrue
l32_mlp_str1.00
l32_post_normtrue
l32_post_norm_str1.00
l33_input_normtrue
l33_input_norm_str1.00
l33_attntrue
l33_attn_str1.00
l33_attn_normtrue
l33_attn_norm_str1.00
l33_mlptrue
l33_mlp_str1.00
l33_post_normtrue
l33_post_norm_str1.00
l34_input_normtrue
l34_input_norm_str1.00
l34_attntrue
l34_attn_str1.00
l34_attn_normtrue
l34_attn_norm_str1.00
l34_mlptrue
l34_mlp_str1.00
l34_post_normtrue
l34_post_norm_str1.00
l35_input_normtrue
l35_input_norm_str1.00
l35_attntrue
l35_attn_str1.00
l35_attn_normtrue
l35_attn_norm_str1.00
l35_mlptrue
l35_mlp_str1.00
l35_post_normtrue
l35_post_norm_str1.00
final_normtrue
final_norm_str1.00

What it is

This is the deepest control this pack offers over how FLUX Klein 9B reads your prompt - not per-layer, but per functional sub-component inside each of the encoder's 36 layers: the input norm, the attention block, the attention's own norm, the MLP, and the post-norm, each independently scalable. That's 5 controls x 36 layers, plus embed_tokens and a final_norm, for 182 total dials. Klein's text encoder is Qwen3-8B, a genuine language model reading your prompt as an instruction, and the community has already documented real, specific problems that live at exactly this level of granularity - identity drift on edits, and a quality gap the community itself argues about between the fp8 and full-precision versions of this same encoder. This node is for chasing problems that specific: not "the LoRA is too strong," but "this one attention block is doing something I don't want."

Be honest with yourself about when you need this. It's a scalpel for a narrow class of problem - most prompt issues on Klein are fixed by writing plainer, more direct sentences (Klein reads instructions, not tag soup) long before you need to touch a sub-component weight.

How it works

Every sub-component gets a strength multiplier: 1.0 is unchanged, below 1.0 weakens it, above 1.0 boosts it. It patches through ComfyUI's own add_patches system, which the pack notes is LoRA-safe - your LoRAs still apply correctly stacked on top of whatever you change here. One real constraint worth knowing: the underlying model ships partly quantized, and only the float tensors (norms, some weights) are actually modifiable; the quantized layers (U8, F8_E4M3) stay untouched regardless of what you set here.

The recommended workflow, per the pack itself: run Text Encoder Inspector (Qwen3-8B / Flux 2 Klein) first to find which sub-components actually respond to your specific prompt, then come here and adjust just those, rather than guessing across 182 controls blind.

Inputs and outputs that matter

  • clip - the Qwen3-8B encoder.
  • preset - quick starting points like "Weaken ALL attn 90%," "Weaken ALL mlp 85%," "Global 95%," and more, alongside Custom and Default.
  • embed_tokens / embed_tokens_str and final_norm / final_norm_str - the two global controls; the pack's own tip is that boosting embed_tokens to 1.5–2.0 tends to improve prompt adherence.
  • l0_input_norm through l35_post_norm (five sub-components per layer, 36 layers) - every one individually toggleable with its own strength slider.
  • save_model, save_mode (full_model, diff_only, or both), filename - export your modified weights to disk instead of applying them live every run.

Outputs: clip (the patched encoder) and info (a summary of what got modified).

Installing it

No external backend required - this node works the moment the pack is installed. ComfyUI Manager (search "Realtime LoRA Trainer") or:

cd ComfyUI/custom_nodes
git clone https://github.com/ShootTheSound/comfyUI-Realtime-Lora

Restart ComfyUI.

Common issues

Setting a strength to 0.0 produces noise or broken output. Expected - the pack warns about this directly. Stick to roughly 0.5–1.5 for real adjustments; 0.0 is a diagnostic extreme, not a usable setting.

Nothing changes despite adjusting a sub-component. Check whether you're touching a quantized layer - U8 and F8_E4M3 tensors in this model are preserved unchanged regardless of the slider, by design, since only float tensors are actually patchable.

182 controls is too many to reason about without a starting point. This is exactly what the Inspector node is for - run it first, read which sub-components actually activated for your prompt, and adjust only those rather than sweeping through every layer.

Trying to use this to fix an image-quality problem, not a text-comprehension one. This node reshapes how the encoder reads and weighs your prompt - it has no direct control over the diffusion model or the VAE. If the problem is in generated pixels rather than prompt following, look at FLUX Model Layer Editor or VAE Deep Debiaser territory instead.

Categorymodel_patches

Inputs (369)

NameTypeDefaultDescription
clipCLIPCLIP/Text Encoder model (Qwen3-8B for Flux 2 Klein)
presetCOMBODefault13 options: Custom, Default, Weaken ALL attn 90%, Weaken ALL attn 85%, Weaken ALL attn 80%, Weaken ALL mlp 90%, +7
save_modelBOOLEANfalseSave the modified model to disk when enabled
save_modeCOMBOfull_modelfull_model = complete safetensors. diff_only = only changed weights (small). both = save both files.
filenameSTRINGautoFilename (no extension). 'auto' = generate from settings.
embed_tokensBOOLEANtrue
embed_tokens_strFLOAT1.00-5–5
l0_input_normBOOLEANtrue
l0_input_norm_strFLOAT1.00-5–5
l0_attnBOOLEANtrue
l0_attn_strFLOAT1.00-5–5
l0_attn_normBOOLEANtrue
l0_attn_norm_strFLOAT1.00-5–5
l0_mlpBOOLEANtrue
l0_mlp_strFLOAT1.00-5–5
l0_post_normBOOLEANtrue
l0_post_norm_strFLOAT1.00-5–5
l1_input_normBOOLEANtrue
l1_input_norm_strFLOAT1.00-5–5
l1_attnBOOLEANtrue
l1_attn_strFLOAT1.00-5–5
l1_attn_normBOOLEANtrue
l1_attn_norm_strFLOAT1.00-5–5
l1_mlpBOOLEANtrue
l1_mlp_strFLOAT1.00-5–5
l1_post_normBOOLEANtrue
l1_post_norm_strFLOAT1.00-5–5
l2_input_normBOOLEANtrue
l2_input_norm_strFLOAT1.00-5–5
l2_attnBOOLEANtrue
l2_attn_strFLOAT1.00-5–5
l2_attn_normBOOLEANtrue
l2_attn_norm_strFLOAT1.00-5–5
l2_mlpBOOLEANtrue
l2_mlp_strFLOAT1.00-5–5
l2_post_normBOOLEANtrue
l2_post_norm_strFLOAT1.00-5–5
l3_input_normBOOLEANtrue
l3_input_norm_strFLOAT1.00-5–5
l3_attnBOOLEANtrue
l3_attn_strFLOAT1.00-5–5
l3_attn_normBOOLEANtrue
l3_attn_norm_strFLOAT1.00-5–5
l3_mlpBOOLEANtrue
l3_mlp_strFLOAT1.00-5–5
l3_post_normBOOLEANtrue
l3_post_norm_strFLOAT1.00-5–5
l4_input_normBOOLEANtrue
l4_input_norm_strFLOAT1.00-5–5
l4_attnBOOLEANtrue
l4_attn_strFLOAT1.00-5–5
l4_attn_normBOOLEANtrue
l4_attn_norm_strFLOAT1.00-5–5
l4_mlpBOOLEANtrue
l4_mlp_strFLOAT1.00-5–5
l4_post_normBOOLEANtrue
l4_post_norm_strFLOAT1.00-5–5
l5_input_normBOOLEANtrue
l5_input_norm_strFLOAT1.00-5–5
l5_attnBOOLEANtrue
l5_attn_strFLOAT1.00-5–5
l5_attn_normBOOLEANtrue
l5_attn_norm_strFLOAT1.00-5–5
l5_mlpBOOLEANtrue
l5_mlp_strFLOAT1.00-5–5
l5_post_normBOOLEANtrue
l5_post_norm_strFLOAT1.00-5–5
l6_input_normBOOLEANtrue
l6_input_norm_strFLOAT1.00-5–5
l6_attnBOOLEANtrue
l6_attn_strFLOAT1.00-5–5
l6_attn_normBOOLEANtrue
l6_attn_norm_strFLOAT1.00-5–5
l6_mlpBOOLEANtrue
l6_mlp_strFLOAT1.00-5–5
l6_post_normBOOLEANtrue
l6_post_norm_strFLOAT1.00-5–5
l7_input_normBOOLEANtrue
l7_input_norm_strFLOAT1.00-5–5
l7_attnBOOLEANtrue
l7_attn_strFLOAT1.00-5–5
l7_attn_normBOOLEANtrue
l7_attn_norm_strFLOAT1.00-5–5
l7_mlpBOOLEANtrue
l7_mlp_strFLOAT1.00-5–5
l7_post_normBOOLEANtrue
l7_post_norm_strFLOAT1.00-5–5
l8_input_normBOOLEANtrue
l8_input_norm_strFLOAT1.00-5–5
l8_attnBOOLEANtrue
l8_attn_strFLOAT1.00-5–5
l8_attn_normBOOLEANtrue
l8_attn_norm_strFLOAT1.00-5–5
l8_mlpBOOLEANtrue
l8_mlp_strFLOAT1.00-5–5
l8_post_normBOOLEANtrue
l8_post_norm_strFLOAT1.00-5–5
l9_input_normBOOLEANtrue
l9_input_norm_strFLOAT1.00-5–5
l9_attnBOOLEANtrue
l9_attn_strFLOAT1.00-5–5
l9_attn_normBOOLEANtrue
l9_attn_norm_strFLOAT1.00-5–5
l9_mlpBOOLEANtrue
l9_mlp_strFLOAT1.00-5–5
l9_post_normBOOLEANtrue
l9_post_norm_strFLOAT1.00-5–5
l10_input_normBOOLEANtrue
l10_input_norm_strFLOAT1.00-5–5
l10_attnBOOLEANtrue
l10_attn_strFLOAT1.00-5–5
l10_attn_normBOOLEANtrue
l10_attn_norm_strFLOAT1.00-5–5
l10_mlpBOOLEANtrue
l10_mlp_strFLOAT1.00-5–5
l10_post_normBOOLEANtrue
l10_post_norm_strFLOAT1.00-5–5
l11_input_normBOOLEANtrue
l11_input_norm_strFLOAT1.00-5–5
l11_attnBOOLEANtrue
l11_attn_strFLOAT1.00-5–5
l11_attn_normBOOLEANtrue
l11_attn_norm_strFLOAT1.00-5–5
l11_mlpBOOLEANtrue
l11_mlp_strFLOAT1.00-5–5
l11_post_normBOOLEANtrue
l11_post_norm_strFLOAT1.00-5–5
l12_input_normBOOLEANtrue
l12_input_norm_strFLOAT1.00-5–5
l12_attnBOOLEANtrue
l12_attn_strFLOAT1.00-5–5
l12_attn_normBOOLEANtrue
l12_attn_norm_strFLOAT1.00-5–5
l12_mlpBOOLEANtrue
l12_mlp_strFLOAT1.00-5–5
l12_post_normBOOLEANtrue
l12_post_norm_strFLOAT1.00-5–5
l13_input_normBOOLEANtrue
l13_input_norm_strFLOAT1.00-5–5
l13_attnBOOLEANtrue
l13_attn_strFLOAT1.00-5–5
l13_attn_normBOOLEANtrue
l13_attn_norm_strFLOAT1.00-5–5
l13_mlpBOOLEANtrue
l13_mlp_strFLOAT1.00-5–5
l13_post_normBOOLEANtrue
l13_post_norm_strFLOAT1.00-5–5
l14_input_normBOOLEANtrue
l14_input_norm_strFLOAT1.00-5–5
l14_attnBOOLEANtrue
l14_attn_strFLOAT1.00-5–5
l14_attn_normBOOLEANtrue
l14_attn_norm_strFLOAT1.00-5–5
l14_mlpBOOLEANtrue
l14_mlp_strFLOAT1.00-5–5
l14_post_normBOOLEANtrue
l14_post_norm_strFLOAT1.00-5–5
l15_input_normBOOLEANtrue
l15_input_norm_strFLOAT1.00-5–5
l15_attnBOOLEANtrue
l15_attn_strFLOAT1.00-5–5
l15_attn_normBOOLEANtrue
l15_attn_norm_strFLOAT1.00-5–5
l15_mlpBOOLEANtrue
l15_mlp_strFLOAT1.00-5–5
l15_post_normBOOLEANtrue
l15_post_norm_strFLOAT1.00-5–5
l16_input_normBOOLEANtrue
l16_input_norm_strFLOAT1.00-5–5
l16_attnBOOLEANtrue
l16_attn_strFLOAT1.00-5–5
l16_attn_normBOOLEANtrue
l16_attn_norm_strFLOAT1.00-5–5
l16_mlpBOOLEANtrue
l16_mlp_strFLOAT1.00-5–5
l16_post_normBOOLEANtrue
l16_post_norm_strFLOAT1.00-5–5
l17_input_normBOOLEANtrue
l17_input_norm_strFLOAT1.00-5–5
l17_attnBOOLEANtrue
l17_attn_strFLOAT1.00-5–5
l17_attn_normBOOLEANtrue
l17_attn_norm_strFLOAT1.00-5–5
l17_mlpBOOLEANtrue
l17_mlp_strFLOAT1.00-5–5
l17_post_normBOOLEANtrue
l17_post_norm_strFLOAT1.00-5–5
l18_input_normBOOLEANtrue
l18_input_norm_strFLOAT1.00-5–5
l18_attnBOOLEANtrue
l18_attn_strFLOAT1.00-5–5
l18_attn_normBOOLEANtrue
l18_attn_norm_strFLOAT1.00-5–5
l18_mlpBOOLEANtrue
l18_mlp_strFLOAT1.00-5–5
l18_post_normBOOLEANtrue
l18_post_norm_strFLOAT1.00-5–5
l19_input_normBOOLEANtrue
l19_input_norm_strFLOAT1.00-5–5
l19_attnBOOLEANtrue
l19_attn_strFLOAT1.00-5–5
l19_attn_normBOOLEANtrue
l19_attn_norm_strFLOAT1.00-5–5
l19_mlpBOOLEANtrue
l19_mlp_strFLOAT1.00-5–5
l19_post_normBOOLEANtrue
l19_post_norm_strFLOAT1.00-5–5
l20_input_normBOOLEANtrue
l20_input_norm_strFLOAT1.00-5–5
l20_attnBOOLEANtrue
l20_attn_strFLOAT1.00-5–5
l20_attn_normBOOLEANtrue
l20_attn_norm_strFLOAT1.00-5–5
l20_mlpBOOLEANtrue
l20_mlp_strFLOAT1.00-5–5
l20_post_normBOOLEANtrue
l20_post_norm_strFLOAT1.00-5–5
l21_input_normBOOLEANtrue
l21_input_norm_strFLOAT1.00-5–5
l21_attnBOOLEANtrue
l21_attn_strFLOAT1.00-5–5
l21_attn_normBOOLEANtrue
l21_attn_norm_strFLOAT1.00-5–5
l21_mlpBOOLEANtrue
l21_mlp_strFLOAT1.00-5–5
l21_post_normBOOLEANtrue
l21_post_norm_strFLOAT1.00-5–5
l22_input_normBOOLEANtrue
l22_input_norm_strFLOAT1.00-5–5
l22_attnBOOLEANtrue
l22_attn_strFLOAT1.00-5–5
l22_attn_normBOOLEANtrue
l22_attn_norm_strFLOAT1.00-5–5
l22_mlpBOOLEANtrue
l22_mlp_strFLOAT1.00-5–5
l22_post_normBOOLEANtrue
l22_post_norm_strFLOAT1.00-5–5
l23_input_normBOOLEANtrue
l23_input_norm_strFLOAT1.00-5–5
l23_attnBOOLEANtrue
l23_attn_strFLOAT1.00-5–5
l23_attn_normBOOLEANtrue
l23_attn_norm_strFLOAT1.00-5–5
l23_mlpBOOLEANtrue
l23_mlp_strFLOAT1.00-5–5
l23_post_normBOOLEANtrue
l23_post_norm_strFLOAT1.00-5–5
l24_input_normBOOLEANtrue
l24_input_norm_strFLOAT1.00-5–5
l24_attnBOOLEANtrue
l24_attn_strFLOAT1.00-5–5
l24_attn_normBOOLEANtrue
l24_attn_norm_strFLOAT1.00-5–5
l24_mlpBOOLEANtrue
l24_mlp_strFLOAT1.00-5–5
l24_post_normBOOLEANtrue
l24_post_norm_strFLOAT1.00-5–5
l25_input_normBOOLEANtrue
l25_input_norm_strFLOAT1.00-5–5
l25_attnBOOLEANtrue
l25_attn_strFLOAT1.00-5–5
l25_attn_normBOOLEANtrue
l25_attn_norm_strFLOAT1.00-5–5
l25_mlpBOOLEANtrue
l25_mlp_strFLOAT1.00-5–5
l25_post_normBOOLEANtrue
l25_post_norm_strFLOAT1.00-5–5
l26_input_normBOOLEANtrue
l26_input_norm_strFLOAT1.00-5–5
l26_attnBOOLEANtrue
l26_attn_strFLOAT1.00-5–5
l26_attn_normBOOLEANtrue
l26_attn_norm_strFLOAT1.00-5–5
l26_mlpBOOLEANtrue
l26_mlp_strFLOAT1.00-5–5
l26_post_normBOOLEANtrue
l26_post_norm_strFLOAT1.00-5–5
l27_input_normBOOLEANtrue
l27_input_norm_strFLOAT1.00-5–5
l27_attnBOOLEANtrue
l27_attn_strFLOAT1.00-5–5
l27_attn_normBOOLEANtrue
l27_attn_norm_strFLOAT1.00-5–5
l27_mlpBOOLEANtrue
l27_mlp_strFLOAT1.00-5–5
l27_post_normBOOLEANtrue
l27_post_norm_strFLOAT1.00-5–5
l28_input_normBOOLEANtrue
l28_input_norm_strFLOAT1.00-5–5
l28_attnBOOLEANtrue
l28_attn_strFLOAT1.00-5–5
l28_attn_normBOOLEANtrue
l28_attn_norm_strFLOAT1.00-5–5
l28_mlpBOOLEANtrue
l28_mlp_strFLOAT1.00-5–5
l28_post_normBOOLEANtrue
l28_post_norm_strFLOAT1.00-5–5
l29_input_normBOOLEANtrue
l29_input_norm_strFLOAT1.00-5–5
l29_attnBOOLEANtrue
l29_attn_strFLOAT1.00-5–5
l29_attn_normBOOLEANtrue
l29_attn_norm_strFLOAT1.00-5–5
l29_mlpBOOLEANtrue
l29_mlp_strFLOAT1.00-5–5
l29_post_normBOOLEANtrue
l29_post_norm_strFLOAT1.00-5–5
l30_input_normBOOLEANtrue
l30_input_norm_strFLOAT1.00-5–5
l30_attnBOOLEANtrue
l30_attn_strFLOAT1.00-5–5
l30_attn_normBOOLEANtrue
l30_attn_norm_strFLOAT1.00-5–5
l30_mlpBOOLEANtrue
l30_mlp_strFLOAT1.00-5–5
l30_post_normBOOLEANtrue
l30_post_norm_strFLOAT1.00-5–5
l31_input_normBOOLEANtrue
l31_input_norm_strFLOAT1.00-5–5
l31_attnBOOLEANtrue
l31_attn_strFLOAT1.00-5–5
l31_attn_normBOOLEANtrue
l31_attn_norm_strFLOAT1.00-5–5
l31_mlpBOOLEANtrue
l31_mlp_strFLOAT1.00-5–5
l31_post_normBOOLEANtrue
l31_post_norm_strFLOAT1.00-5–5
l32_input_normBOOLEANtrue
l32_input_norm_strFLOAT1.00-5–5
l32_attnBOOLEANtrue
l32_attn_strFLOAT1.00-5–5
l32_attn_normBOOLEANtrue
l32_attn_norm_strFLOAT1.00-5–5
l32_mlpBOOLEANtrue
l32_mlp_strFLOAT1.00-5–5
l32_post_normBOOLEANtrue
l32_post_norm_strFLOAT1.00-5–5
l33_input_normBOOLEANtrue
l33_input_norm_strFLOAT1.00-5–5
l33_attnBOOLEANtrue
l33_attn_strFLOAT1.00-5–5
l33_attn_normBOOLEANtrue
l33_attn_norm_strFLOAT1.00-5–5
l33_mlpBOOLEANtrue
l33_mlp_strFLOAT1.00-5–5
l33_post_normBOOLEANtrue
l33_post_norm_strFLOAT1.00-5–5
l34_input_normBOOLEANtrue
l34_input_norm_strFLOAT1.00-5–5
l34_attnBOOLEANtrue
l34_attn_strFLOAT1.00-5–5
l34_attn_normBOOLEANtrue
l34_attn_norm_strFLOAT1.00-5–5
l34_mlpBOOLEANtrue
l34_mlp_strFLOAT1.00-5–5
l34_post_normBOOLEANtrue
l34_post_norm_strFLOAT1.00-5–5
l35_input_normBOOLEANtrue
l35_input_norm_strFLOAT1.00-5–5
l35_attnBOOLEANtrue
l35_attn_strFLOAT1.00-5–5
l35_attn_normBOOLEANtrue
l35_attn_norm_strFLOAT1.00-5–5
l35_mlpBOOLEANtrue
l35_mlp_strFLOAT1.00-5–5
l35_post_normBOOLEANtrue
l35_post_norm_strFLOAT1.00-5–5
final_normBOOLEANtrue
final_norm_strFLOAT1.00-5–5

Outputs (2)

NameTypeDescription
clipCLIPCLIP with per-sub-component modifications (LoRA-safe)
infoSTRINGText summary of modifications