TE LoRA Extract (Knee)

Every LoRA extractor in this pack makes you answer one question: what rank? This one answers it for you. Feed it a fine-tuned text encoder and its base, and it looks at each layer's singular value curve, finds where the curve elbows over - the "knee" - and cuts the rank there. The full rank-selection menu in ModelUtils' TE extractors is the single most useful default to reach for when you don't want to think about rank at all.

Here's the idea. When you decompose the A - B weight delta with SVD, you get a descending list of singular values. Early ones carry real signal; the tail is numeric dust. Plot that curve and there's usually a visible bend where "meaningful" turns into "noise." Knee detection finds that bend automatically and keeps everything above it. The node computes the full picture per layer, so each layer gets its own natural rank - attention layers that need more capacity get more, layers with nothing going on get almost nothing.

You get two flavors via knee_method:

• sv_knee (default) - finds the elbow on the raw singular value curve, the classic "bend in the spectrum."
• sv_cumulative_knee - finds the elbow on the cumulative curve instead, which behaves differently when a few big singular values dominate the rest.

The inputs that matter

• model_a / model_b - fine-tuned and base encoders from models/text_encoders.
• knee_method - above.
• linear_max_rank / conv_max_rank - caps (default 128) that keep the automatic rank from ever running away on a weird layer. The knee detector can't exceed these.
• knee_probe_offset - extra singular values probed beyond the cap so the detector doesn't mistake the edge of the truncated spectrum for a knee. If your ranks keep landing suspiciously exactly at the cap, bump this.
• output_filename / save_dtype / device / lazy_load / force_clear_cache - the shared family knobs: where the LoRA lands (your loras folder), save precision (fp16 default), compute device with per-layer CPU OOM fallback, and memory streaming.

The result is a normal PEFT-style LoRA (lora_A/lora_B keys) you load with a regular LoraLoader. It's an output node - output_path just tells you where the file went.

Install

ComfyUI Manager, search "Model Utility Toolkit". Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils
cd ComfyUI-ModelUtils
pip install -r requirements.txt

Restart and it appears under ModelUtils/LoRA Extract (TE). Runtime dependency that matters here: unifiedefficientloader, which streams the two safetensors so memory stays flat.

Gotchas

Same family traps apply. mismatch_mode: skip is the default, so a wrong base quietly skips everything and hands you a useless little file. And because this mode runs the full SVD (it needs the whole spectrum to find the knee), it's among the slower extractors - give it a few minutes and a progress bar. If the automatic rank ever underdelivers, the honest fix isn't more knobs, it's falling back to the Fixed node with a rank you chose. But for the common case - "just extract it, I don't want to tune" - Knee is the one.