DoRA Learned Extract (Fixed Rank) (WIP)

This one's labeled "(WIP)" right in its own display name, and that's not marketing modesty - take it at face value. It's the first of five "Learned Extract" nodes in this pack, and it's the most straightforward: instead of just running an SVD on the weight diff between two checkpoints and calling it done, this node uses that SVD as a starting point and then runs actual gradient descent on top of it, directly minimizing the error between the extracted low-rank approximation and the real weight delta.

Here's why that's worth doing at all. Plain SVD gives you the mathematically optimal low-rank approximation of a matrix - optimal in the Frobenius-norm sense, per layer, in isolation. That's a real guarantee, but it's a narrow one: it doesn't account for how DoRA's magnitude-plus-direction decomposition actually behaves once loaded, and it treats every layer independently rather than jointly. Refining the SVD-initialized adapter with an optimizer that directly minimizes reconstruction loss can, in principle, do better against the thing you actually care about than the closed-form SVD answer alone. Whether it reliably does in practice is exactly what "(WIP)" is telling you nobody's fully verified yet.

Why "Fixed" specifically. Of the five Learned nodes in this pack, this is the one where you pick the rank yourself up front rather than letting the extraction target a percentage or an automatically-detected knee. That's the right choice when you already know what rank you want - matching an existing LoRA's size, hitting a specific file-size budget, or just having a strong prior from experience with this particular model pair.

Inputs that matter. model_a (finetune) and model_b (base) - same A-minus-B convention as every extraction node in this pack. linear_dim (default 64) and conv_dim (default 32) are the fixed ranks for linear/attention layers and convolutional layers respectively - every layer gets exactly this rank, no per-layer variation. svd_niter (default 2) controls how many SVD iterations run to initialize the low-rank factors before gradient descent takes over - this is unique to the Fixed variant among the Learned nodes.

The optimization itself has real depth: optimize_iters (default 500, 0 skips optimization and falls back to plain SVD) sets how many gradient steps run. optimizer (default "prodigy") picks the algorithm - Prodigy auto-tunes its own learning rate and is the community's default adaptive choice for this kind of problem, which is presumably why it's the default here too. learning_rate (default 1) matters a lot depending on which optimizer you pick - the tooltip is explicit that AdamW wants something closer to 0.01 while Prodigy wants closer to 1.0, so don't reuse one value across both. lr_schedule (default "plateau") with lr_patience (default 2), lr_factor (default 0.9), and lr_cooldown (default 2) control how the learning rate decays when progress stalls. Three early-stopping conditions - early_stop_loss (default 1e-6), early_stop_stall (default 2000 iterations with no improvement), and early_stop_lr (default 9.01e-9) - keep it from grinding through the full 500 iterations once it's clearly converged or clearly stuck.

The rest is shared plumbing: chunk_large_layers, clamp_quantile (default 0.99), min_diff (default 0), mismatch_mode (default "skip"), skip_patterns/glob_skip_patterns, output_filename (default "extracted_lora"), save_dtype (default fp16), device (default cuda), lazy_load (on by default), and force_clear_cache (on by default).

Output. output_path (STRING). Output node - runs as a terminal step regardless of downstream wiring.

Installing it. ComfyUI Manager, search "Model Utility Toolkit," or:

cd ComfyUI/custom_nodes
git clone https://github.com/silveroxides/ComfyUI-ModelUtils

then restart.

Troubleshooting. Given the WIP label, the most useful advice is to actually verify the output rather than trust it - load the extracted adapter and compare against the finetune directly before you rely on it for anything. If optimize_iters runs the full 500 without hitting an early-stop condition, that's a sign the learning rate or optimizer choice isn't suited to your model pair; try dropping learning_rate if you switched to AdamW and left it at Prodigy's default of 1, since that's the single most common mismatch this schema invites. And remember optimize_iters: 0 is a legitimate, faster fallback to plain SVD if the gradient-descent refinement isn't paying off for your case - that's effectively what DoRAExtractKnee and its siblings already do without the extra compute cost.