AutoMultiGPU - DisTorch Engine (Layer Sharding)

DisTorch is the layer-sharding approach to multi-GPU: instead of putting whole components on separate cards, you slice the transformer itself - layers 0–18 on GPU0, 19–37 on GPU1 - so a 20B+ DiT's weights physically straddle two GPUs. The DisTorch Engine in this pack is its wrapper: it loads a UNet/DiT (safetensors or GGUF), splits it across the cards, and routes CLIP and VAE onto the secondary GPU. That's the fastest way to get a model that outgrows one card onto a two-card rig.

But here's the thing you need to know before you build anything around it: the actual sharding isn't implemented in this repo. The engine tries to import register_patched_safetensor_modelpatcher from custom_nodes.comfyui_multigpu.distorch_2 - that's ComfyUI-MultiGPU by pollockjj, the project that invented DisTorch. If it's not installed, the import silently fails (a debug-level log you'll never see) and this engine falls back to doing exactly what the PyTorch engine does: UNet on GPU0, CLIP on GPU1, no layer split at all. The README never mentions this dependency, and it's the single most important gotcha in the entire pack.

How it works

On load it tries to register the DisTorch2 patcher (needs ComfyUI-MultiGPU), then loads the UNet onto GPU0 - the patcher is what actually redistributes blocks across devices during inference. sharding_ratio sets the split: 50/50 (default), 60/40, or 40/60, so you can favor whichever card has more VRAM. force_distorch_patch defaults to true and exists to enforce the patcher. CLIP and VAE land on cuda:1.

The inputs that matter

unet_name, clip_name1, clip_name2, vae_name, model_type (auto guesses from the filename), and sharding_ratio. Optional weight_dtype (fp8 family included) and force_distorch_patch. Outputs: MODEL, CLIP, VAE.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/nexusfinancial-dev/ComfyUI-AutoMultiGPU.git
cd ComfyUI-AutoMultiGPU
pip install -r requirements.txt

And then - because the source demands it - also:

cd ComfyUI/custom_nodes
git clone https://github.com/pollockjj/ComfyUI-MultiGPU.git

Plus ComfyUI-GGUF if you load .gguf files.

Where people get burned

Beyond the missing-dependency trap: sharding only pays off if your two cards can actually talk fast. DisTorch's own benchmarks make this brutally clear - on a consumer motherboard splitting PCIe lanes (x8/x8 or x16/x4), a second GPU can be slower for image generation than just offloading to CPU RAM, because every inference step shuttles the sharded layers across the bus. NVLink or dual full x16 slots are the sweet spot; video workloads tolerate slow donors far better than image does. So before you trust the "massive speedup" line in the README, check your PCIe topology. And if you only have one GPU, the DisTorch choice is meaningless - everything collapses onto GPU0 and you've built a slower stock loader.