DoRA Learned Extract (Ratio) (WIP)

The last of the eight DoRA extraction nodes in this pack, and - like its four Learned siblings - it's labeled "(WIP)" in its own display name. Take that seriously: this is the newest and least-verified layer of an already fairly specialist toolset, and the honest move is to check its output against something you trust rather than assume it just works because it ran without errors.

This one pairs DoRAExtractRatio's rank-selection rule with the gradient-descent refinement stage shared across all five Learned nodes. The ratio method itself is the simplest of the three non-learned selection strategies in this pack: instead of accumulating cumulative energy (quantile) or finding a curve elbow (knee), it just keeps any singular value that's still at least 1/ratio the size of the layer's largest one, and drops everything smaller. It's a local, per-value cutoff rather than a global, cumulative one - which means it can behave a bit differently from quantile-based selection on a layer where the singular values don't have one obvious dominant peak, since a flat spread of similarly-sized values either mostly clears the threshold or mostly doesn't, rather than accumulating gracefully toward a target.

What the "Learned" stage adds. SVD gives the mathematically optimal low-rank fit to a weight matrix in isolation, but that optimality is narrow - it doesn't account for DoRA's magnitude-and-direction reconstruction or for how layers interact once the adapter is actually loaded. This node uses the ratio-selected SVD result as a starting point, then runs an optimizer to directly minimize reconstruction loss against the real weight delta, on the theory that it can close some of that gap. Whether it reliably does, for your particular model pair, is exactly the kind of thing "(WIP)" means you should verify rather than assume.

Inputs that matter. model_a (finetune) and model_b (base) set up the diff. linear_ratio and conv_ratio (both default 2, range 1–100) are the actual selection dial - a value of 2 keeps singular values at least half the size of the layer's largest; raising it lowers the bar and keeps more, weaker values. linear_max_rank and conv_max_rank (both default 128) cap rank regardless of what the ratio threshold would otherwise select.

Optimization controls match the rest of the Learned family: optimize_iters (default 500, 0 skips refinement and falls back to plain ratio-based SVD - effectively DoRAExtractRatio for less compute), optimizer (default "prodigy" - adaptive, doesn't need you to hand-tune a learning rate), learning_rate (default 1, correct for Prodigy; the tooltip flags AdamW as wanting closer to 0.01), lr_schedule (default "plateau") shaped by lr_patience (default 2), lr_factor (default 0.9), and lr_cooldown (default 2), and three early-stop conditions - early_stop_loss (1e-6), early_stop_stall (2000 iterations), early_stop_lr (9.01e-9).

Shared plumbing: chunk_large_layers for fused transformer QKV/MLP blocks, clamp_quantile (default 0.99), min_diff (default 0), mismatch_mode (default "skip"), skip_patterns/glob_skip_patterns, output_filename (default "extracted_lora"), save_dtype (default fp16), device (default cuda), lazy_load (on by default), force_clear_cache (on by default).

Output. output_path (STRING). Output node - it executes as a terminal step whether or not that path is wired downstream.

Installing it. ComfyUI Manager, search "Model Utility Toolkit," or:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

then restart.

Troubleshooting. Run this against DoRAExtractRatio on the same model pair and the same ratio settings before trusting the refined version over the plain one - given the WIP label, that comparison is the whole troubleshooting process. If optimize_iters keeps running to the full budget without an early stop, adjust learning_rate for whichever optimizer you've got selected before concluding anything about the model pair itself. And the usual resource note applies here as everywhere in this extraction family: two full checkpoints have to be in memory for the diff, so leave lazy_load and force_clear_cache on unless your hardware has room to spare.