CLIPVisionLoaderDisTorch2MultiGPU

CLIP Vision models are the image-side encoders behind IP-Adapter-style "use this picture as a reference" workflows and any pipeline that needs to turn a reference image into embeddings the diffusion model can attend to. CLIPVisionLoaderDisTorch2MultiGPU is the DisTorch2 twin of core ComfyUI's CLIPVisionLoader: same clip_name, plus the full split-across-devices toolkit instead of a single device pin.

How it works

Same pattern as every DisTorch2 loader in this pack: compute_device, virtual_vram_gb (default 4), donor_device, expert_mode_allocations for exact placement. Like text encoders, CLIP Vision models typically run once - or once per reference image - rather than on every diffusion step, which makes them a relatively low-risk thing to push aggressively off your compute card if you need the space elsewhere.

One detail worth noticing: eject_models defaults to false here, the same pattern as the CLIP DisTorch2 loaders in this pack (as opposed to the UNet, Checkpoint, and ControlNet variants, which default to eject-on). If you're chaining several IP-Adapter-style reference images in one session and notice the vision encoder isn't reloading from scratch each time, that's the intended behavior, not a leak.

Do you actually need this over the plain loader?

Usually not. CLIP Vision files tend to be modest compared to a UNet or a big LLM-style text encoder - most of the time, CLIPVisionLoaderMultiGPU's simple whole-file device pin is all the control you need, and it's one field simpler to set up. Reach for the DisTorch2 split here specifically when you're running a particularly large vision encoder alongside an already-maxed-out compute card and even a whole-file pin to your donor device doesn't leave enough room.

The inputs and outputs that matter

• clip_name - the CLIP Vision file, not a text-CLIP file. The two aren't interchangeable despite the similar node names.
• compute_device, virtual_vram_gb, donor_device, expert_mode_allocations, eject_models - the DisTorch2 controls, as above.
• Output: CLIP_VISION → your IP-Adapter / vision-conditioning nodes.

How to install it

ComfyUI Manager: search ComfyUI-MultiGPU, install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/pollockjj/ComfyUI-MultiGPU

then restart. No extra dependencies.

Common issues & troubleshooting

You probably don't need heavy offload here. CLIP Vision files are usually modest next to a UNet or an LLM-style text encoder - check the actual file size on disk before assuming you need to push virtual_vram_gb up. Most of the time the default is already more than enough.

Wrong file type entirely. A text-CLIP file dropped where a CLIP Vision file is expected errors on load rather than silently working - the two formats aren't interchangeable regardless of the similar naming.

Shape mismatch further downstream. If your IP-Adapter or vision-conditioning node errors about a mismatched shape, that's almost always an incompatibility between the vision encoder and whatever's expecting it, not something this loader's device settings can fix - check that the CLIP Vision model matches what your adapter was trained against.