Remote CLIP Loader

Your text encoder is the boring VRAM hog in every workflow. CLIP isn't huge by 2026 standards, but it still has to load before the sampler can start, it sits in VRAM the whole generation, and on a single card it's competing with the diffusion model for the same memory. The ComfyUI-RemoteTextEncoder pack's answer is blunt: stop loading it on this machine at all. Point a loader at a second GPU, HTTP the prompt embeddings over, keep this GPU for sampling.

RemoteCLIPLoader is the single-encoder version of that, and it's a drop-in replacement for the built-in CLIPLoader. You pick which model the server should encode with, and out comes a standard CLIP object that any normal text-encode node can consume. It's the node you reach for with an SD 1.x or SD 2.x workflow when you have a second machine (or a big box plus a small one) and want the encoder off your main card.

How it works

The loader itself does almost nothing, and that's by design. It doesn't encode - it just packages a server URL, a model name, and a clip_skip value into a CLIP connection object. The actual work happens later, when a text-encode node fires: your prompt goes to the server's /comfy/encode endpoint and the embeddings come back as base64, which ComfyUI decodes into tensors and wraps as CONDITIONING. No network call happens at load time, so the graph runs instantly and fails at encode time if the server is down.

The model_name dropdown is populated live from the server's GET /v1/models endpoint - the same one the server rescans on every call, so models you drop into server/models/ show up without a server restart. There's a genuine gotcha baked in: if the dropdown is stuck on "no models found", the server URL is wrong or the server isn't running. Fix the URL and refresh - the tooltip says F5, the README says press R on the node; either way, refresh after you add a model on the server side.

The inputs that matter

• server_url - base URL of your server, default http://localhost:8288. This is the one you'll actually touch.
• model_name - the dropdown of models the server discovered. Type a name into custom_model instead if what you want isn't listed yet (it overrides the dropdown).
• clip_skip - number of CLIP layers to skip, 1 = no skip. On SD 1.5 anime checkpoints, 2 is a real quality setting. On SDXL it mostly restates a default you already have, so don't treat it as a quality dial.
• timeout - seconds to wait on the HTTP call before giving up. Default 60 is fine for a LAN; bump it if the server is doing a first-time model download.
• api_key - only if you set auth on the server (optional).

The output is one clip socket. Wire it into CLIPTextEncodeRemote, or honestly any standard CLIPTextEncode - the connection object behaves like a normal CLIP.

Installing the pack

The ComfyUI side is almost nothing to install, which is the point of the design:

cd ComfyUI/custom_nodes
git clone https://github.com/rwfsmith/ComfyUI-RemoteTextEncoder
pip install -r requirements.txt

That requirements.txt is just requests and numpy - torch is already a ComfyUI dependency. ComfyUI Manager works too: search for ComfyUI-RemoteTextEncoder and install it. Then set the server URL in rte_config.json in the node folder (otherwise the dropdown stays empty until the first run saves it) and restart.

The server is the heavier half and lives on the GPU machine:

cd ComfyUI-RemoteTextEncoder/server
pip install torch --index-url https://download.pytorch.org/whl/cu121   # or rocm6.0 for AMD
pip install -r requirements.txt
python server.py --port 8288

Server dependencies are the real list: fastapi, uvicorn, transformers, sentencepiece, protobuf, accelerate, safetensors. Models go in server/models/ as an HF model directory or a single .safetensors with a config.json and tokenizer files beside it; any HF-compatible CLIP or T5 encoder works by repo-id.

Common issues

• Empty dropdown - server unreachable or URL wrong. Set rte_config.json, confirm server.py is listening, refresh.
• Slow first request - the server downloads the model from Hugging Face on first use. Subsequent requests hit its cache. This is why timeout exists.
• Server OOM - run it with --dtype fp16 or --dtype bf16; the server README also suggests trimming --model-ttl so models evict sooner.

Honest verdict: this is a niche tool for a genuinely unusual setup - a second GPU with enough VRAM to hold encoders. If you have one card, don't bother; the complexity isn't worth it. If you have two, it's clean, and prompts never leave your own machines (unlike the "remote encoder" nodes that ping third-party APIs).