ClipProj Loader (all-in-one)

ClipProj Loader does exactly what its display name promises: load a small text encoder on the GPU of your choice, apply a learned projection into the big encoder's space, and hand you a CLIP. One node, five widgets, done. It's the pack's namesake, and it's the one the author tells you not to use first.

The honest summary from the README: "If you have a single card, or a tight one, use ComfyUI's own Load CLIP and pass it through ClipProj Apply. That is all." This node exists for multi-GPU machines - it adds two things the stock loader can't do: pick which card the encoder lands on, and pin it there so it's never moved. Both are useful when you have a spare GPU to park an encoder on. Both are actively harmful on a single card, and it was built on a five-GPU machine - the author says that shows.

Why the default is a trap

The mode input defaults to resident, and resident means the encoder is pinned to the card permanently. On a single GPU, that's 4–9 GB taken away from the diffusion model at every sampling step, so the DiT starts paging its own weights instead of just being slightly slower. The numbers are brutal: someone on an 8 GB card reported the stock 32B encoder looking faster than a projected 4B - because ComfyUI unloads the 32B before sampling, while the pinned 4B stayed put.

If you're on one card, do yourself a favor:

Load CLIP (type: krea2)  →  ClipProj Apply  →  the H3 node's clip input

That chain gets the same projection with ComfyUI's own memory management, which offloads the small encoder to RAM once the prompt is encoded. If you must use this node anyway, at least set mode to streaming so the encoder folds back to RAM when it's done.

What it loads

• clip_name - the small encoder, e.g. a Qwen3-VL-4B from models/text_encoders/.
• type - auto reads the checkpoint header and picks the architecture. The useful safety net: it checks for a vision tower, so a text-only Qwen3 (same hidden width, would silently ignore your prompt) gets refused. Override only if auto fails: krea2 = 4B, boogu = 8B, minimax = 32B.
• projection - the dropdown of matrices in models/clip_projections/, plus the <control:zero>, <control:identity>, and <control:random> baselines. Pick a mmh3-4b-… file for a 4B encoder or mmh3-8b-… for an 8B - the node refuses a mismatch.
• device - which GPU receives the encoder (cuda:0, cuda:1, …; cpu as fallback).
• mode - resident (pinned), streaming (loads in one go, folds back to RAM), or dynamic (paged layer by layer, lowest peak usage, slowest).

Output is a single CLIP, the projected model, which wires into the H3 node's clip input just like the stock one - the node returns an object that behaves like the official CLIP, so nothing downstream knows the difference.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/nicolab28/ComfyUI-ClipProj

Restart ComfyUI (Manager: search "ComfyUI-ClipProj"). No requirements.txt, nothing to pip install - torch and ComfyUI's own modules only. Needs ComfyUI 0.31.0+. First launch creates models/clip_projections/ for the matrices, which come from NicoLab28/ClipProj-MiniMax-H3. The v3/v3.1 files need node 0.1.13 or later.

Rough edges

• KeyError: 'W' on a v3 -mlp matrix means your node version is too old - those files carry no linear matrix and earlier releases can't read them. Update the pack; the | residual only log line is expected, not a corrupt download.
• int8 encoders historically only ran in resident mode (the paged path broke the int8 dequantizer). Since 0.1.13 they work in all three, with a warning if you leave resident - fine if you're on a spare card, which is the only place you should be anyway.
• A pinned encoder ignores ComfyUI's global unload by design. That's the pack's ClipProj Free VRAM node's job, and it also means the pin only releases itself on purpose - another reason to prefer streaming on a tight machine.

Use this node when you have a card to spare. Everyone else: the two-node chain up top, and you're done.