Diffusion Model Loader Resident

Stock ComfyUI has a wasteful habit: when you load a .safetensors model it reads the whole file into system RAM on the CPU, then copies the tensors across the PCIe bus to the GPU. Every time you swap models, the old one gets torn down and the new one makes the same slow trip. Diffusion Model Loader Resident is the flagship node of the GPU Resident Loader pack, and its whole point is to shorten that path: read the safetensors header, grab only the UNet keys it needs, and try to load them straight onto the GPU, keeping the finished model resident so the next run doesn't re-pay the ingest cost.

It's called "resident" for a reason - this pack installs startup monkey patches that change how ComfyUI itself loads and unloads models. The node is the KJ-style loader on top of that patched plumbing, so if you already know Kijai's DiffusionModelLoader from KJNodes, the controls will look familiar: it mirrors the same weight dtype, compute dtype, cublas, SageAttention, and fp16-accumulation knobs.

The inputs that matter for most people:

• model_name - pick your diffusion model from ComfyUI/models/diffusion_models.
• weight_dtype - default, fp8_e4m3fn, fp8_e4m3fn_fast, fp8_e5m2, fp16, bf16, or fp32. If the model fits comfortably, leave it alone; if you're chasing VRAM, fp8 is the standard lever.
• compute_dtype - dtype applied to the patcher after the model is built. default is fine until you know you need something else.
• patch_cublaslinear - ComfyUI's cublas performance toggle. Off by default, and off is fine on most cards.
• sage_attention - attention override for SageAttention variants. Here's the trap: SageAttention is not installed by the pack (its only dependency is safetensors>=0.4.3), so leaving this on disabled is the safe default unless you've installed SageAttention yourself.
• enable_fp16_accumulation - sets torch.backends.cuda.matmul.allow_fp16_accumulation. Off unless you're chasing speed on specific hardware.
• Optional extra_state_dict - an absolute path to a second state dict that gets merged into the main one before model detection. Only matching UNet keys are merged.
• Optional policy_override - a residency policy string. Connect Set Global Residency Policy here, not to extra_state_dict. Old graphs wired policy strings into extra_state_dict; there's a shim that still honors that, but new graphs should use the dedicated input.

Output is a single MODEL that wires straight into your sampler, just like any other loader.

The mechanism is worth understanding because it sets expectations. For .safetensors files the node inspects the header, estimates the real byte footprint, and before a GPU-bound load it trims only enough lower-priority residency to cover the estimate plus adaptive headroom (12.5% of the load, 256 MiB floor, 1 GiB ceiling). If the direct GPU read fails it falls back to CPU-read-plus-copy. Re-loading the same file with the same options reuses a live equivalent model instead of rebuilding it. And .ckpt / .pt pickle files? Still CPU-first torch.load() - the fast path is safetensors-only, so hot pickle checkpoints should be converted with the included scripts/convert_checkpoint_to_safetensors.py.

Install it the usual way:

cd ComfyUI/custom_nodes
git clone https://github.com/xmarre/ComfyUI-GPU-Resident-Loader
cd ComfyUI-GPU-Resident-Loader
pip install -r requirements.txt

Or just search "comfyui-gpu-resident-loader" in ComfyUI Manager and restart. For the resident behavior to shine, run ComfyUI with --highvram (or --gpu_only), which makes the pack default to its sticky_gpu policy.

Where people get burned: they expect the fast path on a .ckpt, or they flip on a SageAttention mode without SageAttention installed, or they wire a policy string into extra_state_dict and wonder why the loader thinks it's a file path. Read the node descriptions and you'll dodge all three. If your workflow reloads the same model constantly on a big card, this is the loader you actually want - for a one-off image on a 8 GB card, stock ComfyUI is fine and you don't need any of this.