Auto UNET Loader (MultiGPU - Safetensors & GGUF)

If your model lives as a standalone diffusion file - a diffusion_models-folder checkpoint, a GGUF quant, a FLUX.2/Klein or Wan DiT - the AutoUNETLoaderMultiGPU is the modular drop-in replacement for the stock UNETLoader, with one upgrade: it loads the model onto whichever card actually has the most free VRAM at that moment. Combined with the pack's CLIP and VAE loaders, it's the building block version of the bigger all-in-one engines. You build the pipeline, it handles the placement.

The realistic use case: you've got two cards, one big model, and you're tired of deciding by hand which card should eat it. Set the target to Auto and it checks live free VRAM per GPU and picks the winner. On a single-GPU rig it behaves exactly like a stock loader, which is honestly fine - this is a convenience wrapper, not a magic bullet.

How it works

It scans diffusion_models, unet, and unet_gguf folders into one combined list, resolves the file's path, and loads it through ComfyUI's load_diffusion_model with an explicit load_device. The target_device dropdown decides placement: "Auto (Highest Free VRAM)" sorts the cards by free memory and takes the top; or hard-pick cuda:0, cuda:1, or cpu. The cpu_offload setting then decides what happens to idle weights - auto_smart_offload and always_offload_to_cpu both push inactive weights to system RAM; disabled_vram_only keeps everything in VRAM (faster, riskier). For GGUF files it hands off to ComfyUI-GGUF's loader and patches the patcher onto your chosen device.

The inputs that matter

• unet_name - the model file, safetensors and GGUF mixed in one list.
• weight_dtype - default, the fp8 family (fp8_e4m3fn, fp8_e5m2, fast variant), or fp16/bf16/fp32. fp8 is the easy VRAM half-off move if your card supports it.
• target_device - Auto (highest free VRAM), or explicit.
• cpu_offload - how aggressively to push weights to RAM.

Output is a single MODEL, which feeds your sampler and downstream nodes like any diffusion model.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/nexusfinancial-dev/ComfyUI-AutoMultiGPU.git
cd ComfyUI-AutoMultiGPU
pip install -r requirements.txt

Or ComfyUI Manager → search ComfyUI-AutoMultiGPU. The pack's own deps are just torch, accelerate, safetensors - but GGUF needs ComfyUI-GGUF installed, and that dependency is buried in the code, not the README.

Where people get burned

"Highest free VRAM" is a snapshot in time; the sampler doesn't move the model mid-run, so if the memory picture changes between load and sample, you don't get a rebalance. Also remember CPU offloading is a swap, not a shrink - always_offload_to_cpu keeps VRAM tidy but makes every run slower as weights get shuffled back and forth. And a .gguf pick without ComfyUI-GGUF present will hard-fail with "ComfyUI-GGUF is required," which is the pack's signature gotcha.