TE DoRA Extract (Fixed Rank)

You have a fine-tuned text encoder and the base it came from. You want the difference between them as a lightweight file you can load on top of any compatible base - without retraining anything. That's LoRA extraction, and this node does it with the DoRA variant: instead of a plain low-rank adapter, you get the low-rank factors plus a per-channel magnitude vector, which is what makes DoRA train and behave more like a full fine-tune. It's from the same extraction machinery as the pack's LoRA extractors, just pointed at text encoders and wired for DoRA output.

How it works

Model A is the fine-tuned text encoder, Model B is the base. The node computes A − B per layer, SVD-decomposes that delta into low-rank down/up factors, and - the DoRA part - separates the magnitude direction and stores it as a dora_scale tensor alongside the factors. The result is written to your LoRA directory and loads like any LoRA.

This is the Fixed Rank flavor: you pick linear_dim (default 64) for linear/attention layers and conv_dim (default 32) for conv layers, and every layer of that type gets exactly that rank. Predictable size, predictable shape - but it ignores what the singular value spectrum actually wants. A layer whose delta only needs rank 4 gets rank 64 anyway, which is wasted bytes but never a correctness problem.

Inputs that matter

• model_a / model_b - fine-tuned and base text encoders, both from models/text_encoders. The tooltip spells it out: A - B = LoRA.
• linear_dim / conv_dim - the fixed ranks. Start near the defaults; 64/32 is a sane text-encoder size.
• svd_niter (default 2) - power iterations for the SVD. A couple of extra iterations tighten the low-rank approximation for a modest compute cost; 0–10 range, leave at 2 unless you're chasing quality.
• output_filename (default extracted_te_lora) - written to the LoRA directory, no extension.
• save_dtype (default fp16) - dtype for the extracted factors.
• clamp_quantile (default 0.99) - clamps outlier singular values, which keeps one freak value from dominating the reconstruction.
• min_diff (default 0) - skip layers whose max difference from base is below this; a cheap way to ignore layers that barely changed.
• chunk_large_layers (default off) - split large fused layers (QKV, MLP) into chunks before SVD. Helps if a fused layer blows memory.
• skip_patterns / glob_skip_patterns - skip layers by regex (default) or glob.
• mismatch_mode - skip / zeros / error for missing or incompatible text-encoder tensors.
• lazy_load (default on), force_clear_cache (default off here), device - streaming, cache flushing, and CUDA-with-CPU-fallback.

Outputs

Just output_path - the node is a terminal/output node that reports where the DoRA file landed. It's an output-node, so nothing else is produced; the file is the deliverable.

When to pick Fixed

When you know the rank you want - you're matching a training recipe's dim, or you want every layer to have identical capacity, or you care most about a predictable final size. If you'd rather let the data decide rank per layer, the Frobenius (TextEncoderDoRAExtractFrobenius) and Knee (TextEncoderDoRAExtractKnee) variants from the same pack are the alternatives, and one of them is usually the better first try for a random pair of models.

Install

Part of Model Utility Toolkit (silveroxides/ComfyUI-ModelUtils). ComfyUI Manager → search "Model Utility Toolkit", or:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

Restart ComfyUI. Real dependency: unifiedefficientloader (UEL). Keep ComfyUI current - the pack uses the newer extension API. The pack's extraction code acknowledges kohya-ss/sd-scripts and LyCORIS as its lineage, so if you've used those tools the workflow will feel familiar.