Load Jina IP-Adapter Model

790 megabytes. If you've only ever touched Tencent's stock IP-Adapter weights, which sit in the same folder at a tenth of the size, that number is your first clue this isn't your father's IP-Adapter. The Mugen-Jina-IP-Adapter is 368M parameters trained for five epochs (208,000 steps) at 1024×1024 on the danbooru2024 anime dataset - a real training run aimed at one specific model, CabalResearch/Mugen, not a general-purpose adapter you slap on anything and hope.

What the node does is refreshingly boring: it reads a .safetensors file from ComfyUI/models/ip_adapter/ and hands you a JINA_IP_ADAPTER object for the Apply node. The ipadapter_file dropdown lists whatever it finds there. That's the whole interface - one input, one output.

How it works

The interesting part is in the loading, straight from the code. Loading splits the state dict into two halves. The image_projector.* keys build the projector that turns Jina's vision hidden states into 16 learnable query tokens at SDXL's cross-attention width (2048), using a small attention block with a residual MLP - Q-Former style, the same trick the FaceID Plus line popularized. The unet.* keys are the ip_to_k / ip_to_v projection matrices that attach to eleven cross-attention blocks across the SDXL UNet. The loader keeps both halves around; the Apply node does the actual wiring into a patched model clone.

Because it reads query_tokens, vision_proj, and the UNet key layout out of whatever file you point it at, this node is effectively a generic Jina-style adapter loader. In practice, though, there's exactly one file it's designed for.

Getting the file

curl -L -o ComfyUI/models/ip_adapter/Mugen-Jina-IP-Adapter.safetensors \
  https://huggingface.co/TheRemixer/Mugen-Jina-IP-Adapter/resolve/main/Mugen-Jina-IP-Adapter.safetensors

Same repo also ships WorkflowMugenJinaIPAdapter.json - grab that too and drag it into ComfyUI for a working reference graph. Four previousEpochs checkpoints are up there as well if you want to A/B the training progress (the trainer says results get promising from epoch two).

Install

Shared with the whole pack:

cd ComfyUI/custom_nodes
git clone https://github.com/lRemixl/ComfyUI_JinaCLIP_SDXL_IP_Adapter

then restart, or ComfyUI Manager → search "ComfyUI_JinaCLIP_SDXL_IP_Adapter". There's no requirements.txt in the repo. It imports torch, transformers, safetensors, and einops - ComfyUI ships the first, third, and fourth, so transformers is the one that'll bite:

pip install transformers

Troubleshooting

• Empty dropdown - the file isn't under ComfyUI/models/ip_adapter/ (the node reads that folder, subfolders included).
• State-dict errors at load - you pointed it at the wrong adapter. Stock Tencent IP-Adapter weights don't have the image_projector. prefix this loader expects; it wants the Mugen-Jina-IP-Adapter file specifically.
• Loads fine but Apply logs "keys did not map to recognized block" and nothing happens - same root cause, a file whose unet.* keys don't match SDXL block names. Double-check you downloaded the right safetensors.

Internalize the size before you build a workflow: this is heavyweight next to classic IP-Adapter, and at Apply time the projector moves into VRAM and all the unet weights get shifted to the model device. On an 8GB card you'll feel the difference versus the 100MB adapters you're used to.

One licensing split to plan around: the adapter file itself is Apache 2.0, but the jina-clip-v2 vision model it depends on is CC BY-NC 4.0 (non-commercial). A fully commercial pipeline using this stack doesn't exist yet - read the cards before you ship anything.