DoRA Extract (Quantile)

This is the DoRA extraction node for people who'd rather think in percentages than ranks. Instead of picking a fixed rank per layer or trusting an automatic knee-detector to decide for you, you set a target - "keep enough of each layer to explain 90% of its variation" - and every layer gets whatever rank that actually requires, capped by a ceiling you also control. It's the same idea as this pack's LoRAResizeCumulative node, except that one resizes an existing LoRA and this one extracts a fresh DoRA-format adapter straight from the difference between two full checkpoints.

Extraction like this works because a diff between a base model and its fine-tune, run through SVD layer by layer, is compressible: most of what changed lives in a handful of dominant directions per layer, and the rest is comparatively minor. That's the same mechanism this pack credits kohya-ss/sd-scripts and LyCORIS for popularizing for plain LoRA extraction - this node targets DoRA's format instead, which pairs a low-rank direction update with a magnitude component, trading a slightly more involved output for behavior closer to a true fine-tune than a standard LoRA gives you.

How it works. model_a (your fine-tune) minus model_b (the base) gives a weight delta per layer; SVD decomposes it; instead of a fixed cutoff, the node accumulates singular values until it's captured your target cumulative percentage of the total, then stops. linear_quantile and conv_quantile (both default 0.9, meaning 90%) are the two targets - separate for linear/attention layers versus convolutional ones, because the two layer types often need different amounts of rank to hit the same retention percentage.

Inputs that matter. model_a and model_b are your two checkpoints - finetune minus base equals the extracted adapter. linear_quantile and conv_quantile are the actual dial: push them up toward 1.0 for a more faithful, larger extraction, or down toward 0.5–0.7 if you want a smaller file and can tolerate losing some of the fine-tune's finer effects. linear_max_rank and conv_max_rank (both default 128, up to 16384) are hard caps - useful if a particular layer's diff is complex enough that hitting your quantile target would otherwise demand more rank than you want to pay for.

The remaining fields are the same extraction plumbing this whole node family shares: chunk_large_layers (off by default) for splitting fused QKV/MLP blocks in transformer architectures before extraction, clamp_quantile (default 0.99) to keep outlier singular values from skewing things, min_diff (default 0) to skip layers the fine-tune barely touched, mismatch_mode (default "skip") for layers that don't line up between the two models, skip_patterns (regex by default, or glob with glob_skip_patterns on) to manually exclude layers by name, save_dtype (default fp16), device (default cuda), output_filename (default "extracted_lora"), lazy_load (on by default, streams from disk), and force_clear_cache (on by default, frees CUDA memory per layer).

Output. output_path (STRING). It's an output node, so it runs as a terminal step even without anything downstream reading that path.

Installing it. ComfyUI Manager, search "Model Utility Toolkit," or:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

then restart.

Troubleshooting. The linear_quantile/conv_quantile split trips people up - it's easy to tune one and forget the other, especially since a fine-tune that changed attention behavior heavily but left convolutions mostly alone (or vice versa) really does want different targets for the two. If your output file is much bigger than expected, check linear_max_rank/conv_max_rank aren't set so high that a demanding layer is quietly eating most of your rank budget. And as with any extraction node in this pack, two full checkpoints loaded at once is a real memory cost - leave lazy_load and force_clear_cache on unless you know your hardware can handle both models resident at full precision simultaneously.