IP-Comp Load CLIP Vision

Before any of the IP-Comp nodes can do anything, something has to load an OpenCLIP vision model. That's this node - and it exists because ComfyUI's normal CLIPVisionLoader isn't good enough for this pack. Building a concept subspace needs the text side of CLIP too (a tokenizer), and the standard loader only gives you image encoding. So this pack wraps OpenCLIP directly and passes around its own OPEN_CLIP blob containing the model, the tokenizer, and the preprocessor.

The honest headline: you only need this node on the "create a concept" path. If you're loading a prebuilt subspace and jumping straight to IP-Comp Concept Merge, this node never touches your graph. It exists to feed IP-Comp Concept Subspace, and nothing else.

How it works

Under the hood it's a thin wrapper around OpenCLIP:

model, _, preprocess = open_clip.create_model_and_transforms(f"hf-hub:{hf_name}")
tokenizer = open_clip.get_tokenizer(f"hf-hub:{hf_name}")

It downloads the chosen CLIP weights from HuggingFace Hub on first use, then hands back a dict with the model, tokenizer, preprocess, device, and the HF name. That dict is the OPEN_CLIP type you see on the wire. Nothing runs until IP-Comp Concept Subspace pulls text through it, so the expensive part - encoding your descriptions - happens downstream.

The inputs that matter

There are exactly two, and you'll touch the first constantly and the second rarely:

• clip_vision - a dropdown with ViT-H-14 and ViT-bigG-14. Pick ViT-H-14. It's the one that matches the IP-Adapter ViT-H encoders everyone actually runs, and its weights download is a few hundred MB.
• device - a plain string, default "cuda:0". Note it's not a device picker; it's text you type. If you're on CPU or a second GPU, change it accordingly.

The output is a single OPEN_CLIP, wired into the open_clip_model input of IP-Comp Concept Subspace.

Installing it

Standard pack install:

cd ComfyUI/custom_nodes
git clone https://github.com/godmt/ComfyUI-IP-Composer

restart, and let ComfyUI Manager install the open-clip-torch dependency. First run of this node needs internet for the HF download. It caches locally afterwards.

Where people get burned

• ViT-bigG-14 is enormous. BigG is a multi-billion-parameter model. It'll download a lot and eat VRAM, and nothing in this pack's typical workflow benefits from it. Treat it as an option that exists because OpenCLIP offers it, not because you want it.
• The subspace remembers which model built it. IP-Comp Concept Subspace stamps the HF model name into the subspace's metadata. Load a subspace built with ViT-H and feed it embeddings from a different vision model, and the merge just won't land. If results look broken, check that the whole chain is on one model.
• The device string trips people up. It's free text with a default, not a dropdown of your GPUs. Typo the device and the model load fails on the first run, which reads like a download problem.

It's a utility node, so don't expect drama - but getting the model choice right here is what keeps every downstream node honest.