Offload Anything (GPU/CPU)

ComfyUI's automatic memory management is genuinely good - until your setup stops being "one GPU, one workflow." The scheduler decides which card a model runs on, when to offload it, and when to keep it resident, and on a normal single-card rig that's fine. The moment you've got a second GPU sitting idle, a model that won't fit your main card, or you're trying to run Flux/SD3/Z-Image on CPU, the heuristics guess wrong and every render dies with a cryptic crash.

That's the gap "Offload Anything (GPU/CPU)" - class AnyDeviceOffload - is built for. It's one node, no chain of passthroughs: you feed it a MODEL, CLIP, and/or VAE, tell it where they should live, and it forces them there and keeps them there. Or it nukes them out of VRAM the instant the image is decoded. It's a manual override for ComfyUI's memory scheduler, and for the crash-fixing it does along the way, it's honestly pretty clever.

How it works

The pack patches xformers.ops.memory_efficient_attention globally at import time - you get this whether you use the node or not. That's the invisible part: on GPUs older than Ampere (compute capability < 8.0), xformers often has no kernel and just fails; on CPU it crashes outright with "No operator found." The patch routes attention to PyTorch's scaled dot-product attention instead, casting to float32 on CPU. You don't see any of it - you just get a CPU run that doesn't explode.

Inside the node itself, offload_process does three things. First, it sets the internal load_device / offload_device / current_device flags on the patcher - that's what actually tells ComfyUI where the model "lives." Second, it wraps the diffusion model's forward in a "Runtime Device Guard" that reads the device of every incoming tensor and, if the weights aren't there, moves them on the spot. Self-healing against ComfyUI moving things under you. Third, it wraps VAE decode so latents are cast to the VAE's dtype and device - which is also where the "kill switch" lives.

The inputs that matter

• target_device - the dropdown, built live from what you actually have: cuda:0, cuda:1, mps on Macs, cpu. Pick CPU and safe mode auto-enables (float32 casting, xformers patch).
• vae_mode - Vae Patched (default since v1.0.3) vs Original. Patched forces float32 VAE and auto-casts latents. This is the fix for the classic Flux/SD3 crash, RuntimeError: Input type (c10::BFloat16) and bias type (float) should be the same, plus black images, NaN, and CuDNN "unable to find an engine" errors.
• keep_in_memory - the interesting one. True pins everything on the target device for instant re-runs. False is the kill switch: after decode it pushes the VAE to CPU and runs gc.collect() + torch.cuda.empty_cache().

Outputs are simply MODEL, CLIP, VAE - wire each into your KSampler, CLIPTextEncode, or VAEDecode. You don't have to connect all three; it handles whatever you feed it.

Installing

ComfyUI Manager, search "ComfyUI Any-Device Offload", install, restart. Or the manual way:

cd ComfyUI/custom_nodes/
git clone https://github.com/FearL0rd/ComfyUI-AnyDeviceOffload

Then restart ComfyUI. There are no model downloads and requirements.txt is basically torch, gc, types - everything you already have. You can't really misinstall this one.

Where people get burned

The real gotcha is the kill switch. keep_in_memory=False unloads after the VAE finishes decoding, but if one model feeds multiple KSamplers downstream, the node can't know more sampling is coming - and the graph stalls, waiting on a model that's already been pushed to CPU. That exact report is in the author's own r/comfyui announcement thread, on a 24GB card running an 8-model workflow. If a workflow with split samplers hangs on you, flip keep_in_memory back to True and handle cleanup yourself.

Also: don't mistake "possible" for "fast." VRAM bandwidth is roughly 11x the PCIe bus, so running a 6B model on CPU is a leave-it-overnight proposition. This node makes CPU viable for low-VRAM cards (and people do run Z-Image on CPU with it), it doesn't make it quick. And don't stack it everywhere - one node per device boundary is plenty.