Smart Resident Scheduler (LMSL)

Here's the run that makes you look for this node. You're on a 12GB card, --lowvram (or --enable-dynamic-vram), and your workflow touches four models: base checkpoint, a LoRA stack, a different VAE, then back to the first checkpoint for a second sampler pass. ComfyUI dutifully unloads, loads, unloads, loads - and somewhere in the middle it evicts the checkpoint it is about to need again thirty seconds later. Everything finishes, it's just slow in a way that feels stupid rather than inevitable.

That's thrashing, and the reason it hurts is arithmetic: GPU memory bandwidth is roughly 11x the PCIe bus, so a round trip through system RAM is never cheap. Smart Resident Scheduler (LMSL) exists to make those eviction calls slightly less myopic.

What it actually is

A memory-management plugin, not a model node. There's no checkpoint to download, no pip dependency, and nothing to wire. It ships in a pack called lmsl-smart-offload, and the SmartResidentScheduler node is really just a control panel for a patch that installs itself when ComfyUI starts.

The mechanism is worth understanding, because it's the kind of thing that can quietly do nothing:

At prompt time, the pack hangs a handler on PromptServer's prompt hook. When you hit Queue, it reads the submitted graph, topologically sorts it, and finds every node that produces MODEL/CLIP/VAE and has a file-picker input (ckpt_name, unet_name, lora_name, …). Each becomes a stable key like CheckpointLoaderSimple|wan21_t2v_14B_fp8.safetensors, so the same file used in three places collapses into one entry. From that it derives the ordered list of positions where each model gets used.

At unload time, it wraps comfy.model_management.free_memory - the single place ComfyUI decides what to throw out. Before delegating to the original, it merges into keep_loaded every currently-resident model whose next use distance is within keep_steps. That's a hand-rolled approximation of Belady's optimal eviction: dump the thing needed furthest away, keep the thing needed next. The original function still does the actual unloading, so ComfyUI's own ordering and dynamic-VRAM logic survive.

A VRAM pressure guard turns the whole thing off when free memory is already below minimum_inference_memory(). The author's stated design goal is "never cause an OOM you wouldn't otherwise have," and the code honors it.

The five widgets

The node has no outputs - RETURN_TYPES is empty, so it's a no-op you can't connect to anything. Its only job is to be present so its widgets get read at queue time.

• enabled - master switch. Off means stock ComfyUI behavior.
• keep_steps - the only one you'll actually tune. Default 1 means "don't evict anything I'll need within one topological step." Bump to 2–3 for ping-pong workflows (A → B → A). 0 is the most conservative setting: protect only what's in use right now.
• prefetch_enabled (default off) with prefetch_min_free_mb (2048) - when on, it warms the next planned model during an idle window, but only if free VRAM clears your floor plus the model's offloaded size.
• verbose - flip this first. It logs the plan (plan ready: N model keys, N events) and each load event with its rank.

Installing it

No dependencies at all - the pyproject.toml declares none, and there's no requirements.txt. From the registry:

comfy node install lmsl-smart-offload

Or straight from git:

cd ComfyUI/custom_nodes
git clone https://github.com/mxh678988/lmsl-smart-offload

Then restart. In the startup log you want two lines: patched free_memory / load_models_gpu and on_prompt_handler registered. If the second one is missing, the hook never attached - the pack retries for about ten seconds, then gives up and leaves stock scheduling in place.

Where people get burned

You don't need the node. The patch is global on import. Drop the node in only when you want non-default values for that workflow.

Those values are sticky. The config lives in a module-level object, so a workflow that sets keep_steps=3 leaves that number in place for later runs containing no scheduler node at all. Restart between experiments or you'll be comparing against a setting you forgot.

ComfyUI version drift is the real risk. The wrapper is built against a specific free_memory(memory_required, device, keep_loaded=..., for_dynamic=..., pins_required=..., ram_required=...) signature - the README names the build it was verified on (--enable-dynamic-vram, ComfyUI 0.35.x). On a build whose signature differs, it forwards keywords the original doesn't accept. Failures get logged, but a wrapper in the middle of memory management deserves a source read before you trust it, and with essentially no community discussion of this pack, you're the QA.

It won't rescue a card that doesn't need rescuing. If a single model plus one VAE is all you run, there's nothing to schedule. Test the boring fix first: on cards that barely fit a model, dropping --lowvram-style flags has been measured at roughly a 2x speedup, because the offloading itself was the slowdown. Reach for the scheduler when your graph genuinely swaps models mid-run.