TE LoRA Extract (Fixed Rank)

You have a fine-tuned text encoder sitting on disk, and you'd rather it be a LoRA. That's not a training problem, it's a math problem - and this node does the math. Give it the fine-tuned encoder and the base encoder it was derived from, and it hands you a LoRA file that captures the difference. Same trick the old kohya "Extract LoRA" utility did for checkpoints, done natively in ComfyUI and aimed squarely at text encoders.

Why bother? A full CLIP encoder is hundreds of MB, and a T5/Qwen-class encoder is a serious chunk of disk. A LoRA that holds just the finetune delta is a few MB and drops into any normal LoraLoader. You can also ship the "personality" of a finetuned model's conditioning without shipping the whole model. If that sounds like a tool you'll use once a month, it probably is - but when you need it, there's nothing else in the graph that does it.

How it works

Every .weight tensor in the two encoders is subtracted: A - B, where A is the fine-tuned encoder and B is the base. That per-layer difference is a low-rank-ish matrix, so it's decomposed with SVD into two thin factors - lora_A and lora_B - which is exactly the form ComfyUI's LoRA loader expects. The node writes PEFT-style keys (lora_A.weight, lora_B.weight) with a text_encoders.transformer... prefix so ComfyUI can actually apply them, and it streams the result to disk as it goes, so a huge encoder doesn't blow up your RAM.

Fixed rank means you pick the rank. This is the fastest mode of the pack's extractors because it uses torch's low-rank SVD (svd_lowrank) with power iterations rather than computing the full singular value decomposition of every layer. It's also the most predictable - you know exactly what you're getting.

The inputs that matter

• model_a / model_b - both picked from your models/text_encoders folder. A is the fine-tuned one, B is the base. Same architecture, or you'll get a silently empty file (see below).
• linear_dim / conv_dim - the rank for linear/attention layers (default 64) and convolution layers (default 32). Higher rank = more fidelity, bigger file, more risk of overfitting if you overdo it. 64 is a sane starting point for attention; crank it if the extracted LoRA underdelivers.
• svd_niter - SVD power iterations (default 2). More is more accurate but slower; you'll rarely need more than a couple.
• output_filename - the file name (no extension) for the result, written into your loras folder. Default extracted_te_lora, so change it if you don't want every run overwriting the last one.
• device - cuda by default; if a layer OOMs, the node retries just that layer on CPU rather than dying.

The node is an output node: it saves the file and exposes output_path so you know where it landed. Nothing needs to be wired after it.

Install

ComfyUI Manager is the easy path - search for "Model Utility Toolkit" (the pack's display name, by silveroxides). Or clone it manually:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils
cd ComfyUI-ModelUtils
pip install -r requirements.txt

Restart ComfyUI. The dependency that actually matters for these extract nodes is unifiedefficientloader (the streaming safe-open it uses to read tensors without loading whole models); the rest of requirements.txt (requests, Pillow, mutagen, av) serves other nodes in the pack.

Gotchas

• Wrong base = garbage. If B isn't really the base A was trained from, A - B isn't a clean delta and you'll get a LoRA that does nothing useful. Mismatched or missing tensors are skipped silently by default (mismatch_mode: skip) - a nearly-empty output file is the symptom.
• It's slow, and it's supposed to be. SVD over an entire encoder takes minutes, not seconds. lazy_load (on by default) keeps memory down; force_clear_cache buys VRAM at the cost of speed.
• QKV/MLP fused layers can fail SVD. If individual layers error out, flip on chunk_large_layers and the node splits them into chunks before decomposing.

Want the rank chosen for you instead of guessed? The Knee variant does exactly that - but if you want reproducibility and control, Fixed is the one to reach for.