TE DoRA Extract (Quantile)

DoRA's whole idea - from NVIDIA's "Weight-Decomposed Low-Rank Adaptation" paper - is to split a weight update into magnitude and direction, keep the two apart, and low-rank-approximate only the direction. The text-encoder DoRA extractors in ModelUtils do exactly that, and the quantile variant picks each layer's rank by asking "how much of the singular value mass do I want back?" - 90% by default.

So how's the extraction different from the LoRA flavor? Regular LoRA extraction just subtracts: delta = A - B, then SVD. This node's dora_difference step is sneakier. It rescales each row of the fine-tuned encoder A up to the row magnitude of the base B before subtracting, so the low-rank part captures the direction change, and the per-row magnitude is peeled off into a separate dora_scale tensor. The output file carries lora_A, lora_B, and a dora_scale per layer - the magnitude and direction are literally stored as separate things, which is the DoRA signature.

Honest caveat before you get invested: trained DoRA adapters never really hit critical mass in the community - the per-adapter magnitude term is what breaks stacking, which is the property people actually buy from LoRAs. For extraction specifically, that concern is softer: you're distilling an existing delta, not composing fresh adapters. Still, if you're not sure DoRA matters to you, the LoRA equivalents do the same job with fewer concepts in play.

The inputs that matter

• model_a / model_b - fine-tuned and base encoders from models/text_encoders.
• linear_quantile / conv_quantile - the cumulative singular-value target for linear/attention vs convolution layers (default 0.9). The rank logic is byte-for-byte the same as the LoRA Quantile node - keep enough singular values of the direction delta to hit the target fraction.
• linear_max_rank / conv_max_rank - caps (default 128).
• probe_offset - extra singular values sampled past the cap so the percentile decision is made on a full-enough spectrum.
• output_filename / save_dtype / device / lazy_load / force_clear_cache - the shared family knobs: output name in your loras folder, save precision (fp16 default), compute device with per-layer CPU fallback on OOM, streaming memory.

The single output output_path reports where the file landed; it's an output node, so nothing follows it.

Install

ComfyUI Manager → "Model Utility Toolkit", or:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils
cd ComfyUI-ModelUtils
pip install -r requirements.txt

Restart, and it's under ModelUtils/DoRA Extract (TE). unifiedefficientloader is the dependency doing the streaming work.

Gotchas

The usual silent-skip warning applies (mismatch_mode: skip is default - wrong base, empty file, no error), and this is a full SVD per layer, so expect a real runtime. One DoRA-specific quirk: if you compare against a mismatched base, the direction/magnitude split stops being meaningful, so the "silently wrong" failure is easier to miss than on the LoRA side. And dora_scale support depends on your LoraLoader knowing what to do with it - if you load the file and get nothing, check that your loader handles DoRA keys rather than assuming the extraction failed.