LoRA Extract (Fixed Rank)

If you've got a base checkpoint and a fine-tune of it, you can subtract one from the other and compress the difference into a LoRA - a fraction of the disk space, and something you can apply to other checkpoints in the same family. That's LoRA extraction, and it's a real, well-established technique (kohya-ss's sd-scripts has shipped an implementation of it for years, and it's the acknowledged lineage this whole pack's LoRA tooling draws from). LoRA Extract (Fixed Rank) is silveroxides' ComfyUI-native version of it, and it's the simplest of the five extraction nodes in ComfyUI-ModelUtils - you tell it exactly what rank you want, it gives you exactly that.

The mechanism. model_a is your fine-tuned model, model_b is the base it started from (the node's own tooltips spell this out: "A - B = LoRA"). For every matching layer, the node computes the weight difference, then runs SVD (singular value decomposition) on that difference matrix and keeps the top singular values and vectors as the LoRA's low-rank up/down matrices. Fixed Rank is the version where you pick the rank yourself, up front, applied uniformly - no adaptive logic deciding some layers need more capacity than others. It's the most predictable of the five in output size, and the least adaptive.

Inputs that matter. linear_dim (default 64) and conv_dim (default 32) set the rank for attention/linear layers and conv layers separately - this is the one thing you're actually choosing here. svd_niter (default 2) trades accuracy for speed in the SVD approximation; raise it if extraction quality looks off, at the cost of runtime. clamp_quantile (0.99) clips outlier singular values so one runaway layer doesn't dominate. min_diff skips layers where A and B barely differ, which trims dead weight from the output. skip_patterns (regex, or glob if glob_skip_patterns is on) lets you exclude layers by name entirely - handy for skipping a text encoder you don't want touched. The single output, output_path, is where the extracted .safetensors LoRA lands.

Installing it. ComfyUI Manager, search "Model Utility Toolkit" - or cd ComfyUI/custom_nodes && git clone https://github.com/silveroxides/ComfyUI-ModelUtils and restart. No extra dependencies flagged in the README; you'll need the two checkpoints already sitting in your models folder.

What this actually costs. You're loading two full checkpoints - 2-12GB apiece depending on architecture - and running SVD across every layer that changed. This is not a quick node. device defaults to cuda because SVD on CPU is painfully slow at this scale; lazy_load (on by default) streams tensors from disk instead of holding both full checkpoints in memory at once, and force_clear_cache (also on by default) clears CUDA memory after each layer. Both exist because this is exactly the kind of operation that OOMs on modest cards if you leave them off. If you're on a transformer-heavy architecture with big fused QKV or MLP blocks, chunk_large_layers (off by default) splits those into pieces the SVD step can chew through instead of choking.

Where it bites. model_a and model_b have to be the same architecture - a LoRA extracted between two unrelated models is nonsense, because LoRAs are architecture-bound in general and this is the same constraint showing up at diff time. mismatch_mode defaults to skip, which quietly drops any layer that doesn't line up between A and B rather than erroring - convenient if you expect a few genuinely-added layers, dangerous if you fed it two checkpoints that don't actually share a lineage, because you'll get a LoRA back that looks plausible and is actually missing most of what you wanted. If the rank you picked seems arbitrary and you'd rather the node figure it out per layer, that's what the other four extraction nodes in this pack (Ratio, Quantile, Frobenius, Knee) are for - Fixed Rank is the one to reach for when you specifically want predictable, uniform output size.