TE DoRA Extract (Knee)

Of the three DoRA extractors in this pack, this is the one that wants to do all the thinking. Fixed rank makes you choose; Frobenius makes you choose a fraction. Knee detection looks at each layer's singular value curve and finds the elbow - the point where the spectrum stops being meaningful and turns into noise - and uses that as the rank. It's the "extract me a text-encoder DoRA, you figure out the sizes" button.

How it works

Model A is the fine-tuned text encoder, Model B is the base. Per layer, the node computes A − B, SVD-decomposes the delta, and runs knee detection on the singular value curve to pick the cutoff rank. The DoRA structure - low-rank down/up factors plus the dora_scale magnitude vector - is preserved, and the file lands in your LoRA directory.

Two knobs shape the detection. knee_method (default sv_knee) chooses whether to detect the knee on the raw singular values or on their cumulative distribution - the raw spectrum when you trust the values directly, the cumulative version when you want the energy-accumulation viewpoint. knee_probe_offset (default 32) matters more than it looks: it probes extra singular values beyond the max rank so the detector doesn't mistake the hard cut of a truncated spectrum for a real knee. A false knee at the boundary is the classic failure mode of this whole approach, and the probe offset is the guard against it. linear_max_rank / conv_max_rank (defaults 128) cap the result per layer type, so an uncooperative layer can't explode.

Inputs that matter

• model_a / model_b - fine-tuned and base text encoders (A - B = LoRA), both from models/text_encoders.
• knee_method - sv_knee (raw singular values) or cumulative.
• knee_probe_offset (default 32) - extra singular values sampled past max rank to avoid a false knee.
• linear_max_rank / conv_max_rank (defaults 128) - caps per layer type.
• output_filename (default extracted_te_lora) - written to the LoRA directory.
• save_dtype (default fp16), clamp_quantile (0.99), min_diff (0), chunk_large_layers (off) - same as the other extractors.
• mismatch_mode, skip_patterns / glob_skip_patterns, lazy_load (on), force_clear_cache (off), device - the usual handling.

Outputs

output_path - a terminal node; the file is the deliverable.

When to pick Knee

When you're extracting from a pair you know nothing about and you don't want to babysit rank settings - this is the recommended first pass for a stranger fine-tune. The honest caveat: knee detection is data-dependent and a little black-boxy, and if a layer's spectrum doesn't have a clean elbow you can get a weird rank. If a particular layer comes out suspicious, that's what min_diff (skip near-identical layers) and the max-rank caps are for. The Frobenius variant is the more controllable fallback.

Install

Part of Model Utility Toolkit (silveroxides/ComfyUI-ModelUtils). ComfyUI Manager → search "Model Utility Toolkit", or:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

Restart ComfyUI. Real dependency: unifiedefficientloader (UEL). Keep ComfyUI current - the pack uses the newer extension API. Extraction lineage: kohya-ss/sd-scripts and LyCORIS, per the pack's acknowledgements.