⚡ Text Encoder Layer Streaming (low VRAM)

The second VRAM budget nobody budgets for

By 2026 the text encoder is its own memory problem. You squeeze a 14B DiT into 3GB with streaming or a GGUF quant, and then a T5-XXL or a 7B Qwen encoder ambles in and wants its own 8–14GB before the diffusion model has run a single step. On a small card that's the actual wall, and it's the one this node is built for.

It's the same streaming engine as the pack's DiT node, pointed at ComfyUI's own text encoder implementation. Go into the source and you'll see it walking the model looking for the biggest transformer layer list - layers, block, blocks - and streaming that. T5's encoder.block, Llama/Qwen's model.layers. The weights get moved; the computation stays ComfyUI's. So the conditioning is bit-exact versus not using the node at all, which is the only sane behaviour for a text encoder: you do not want your prompt embedding quietly drifting.

Reach for it when your CLIP loader is loading something big - a 7B-class LLM encoder, T5-XXL - and the model plus encoder together won't sit in VRAM. Anima's own encoder is Qwen3 0.6B, which fits comfortably almost anywhere, so for stock Anima this node is mostly a safety net for when you bolt on a bigger encoder or work at very large resolutions where the latent is eating the budget.

How it works

After a CLIP loader, the node captures the encoder's layer stack into pinned host RAM and keeps resident_layers of them on the GPU permanently. The rest are copied across as the encoder runs, with the same double-buffered overlap the DiT node uses. If free RAM is short, tiering can split the stack: some layers pinned in RAM, the remainder read straight from the model file on disk.

Two things it deliberately does not do: it doesn't quantize anything, and it doesn't touch other clones of the same CLIP. Streaming is armed only on the CLIP object the node returns, so if you've branched a loader into two encoders, only the streamed branch changes behaviour.

Inputs and outputs

• clip - from your CLIP / DualCLIP / text-encoder loader.
• enabled - off passes the CLIP through untouched, which is how you A/B it.
• resident_layers (default 2) - layers parked on the GPU. Higher means fewer transfers and more VRAM. Two is a sane starting point; the encoder is small relative to the diffusion model.
• tiering - Auto (GPU / RAM / SSD) verses RAM (all streamed blocks pinned in RAM). Auto is the default and the right one on a 16GB machine.
• ram_budget_gb - pinned-RAM ceiling for the streamed layers; the remainder comes off disk. 0 means automatic.
• auto_config - on by default, and it overrides the manual numbers based on free VRAM and RAM.

One output: CLIP, wired into your CLIP Text Encode node exactly where the loader used to go.

Install

Manager → WeightFerry → install → restart, or:

cd ComfyUI/custom_nodes
git clone https://github.com/nyanmi-1828/ComfyUI-WeightFerry

Nothing extra to pip install; sageattention and triton-windows are optional and only buy speed. NVIDIA GPU, ComfyUI 0.38.0+, and enough RAM to hold the encoder's file size. The nodes land under WeightFerry/ in the menu.

Where it goes wrong

Quantized text encoders are the trap. The engine copies the encoder's in-memory weights, and it can't do that for a quantized encoder whose weights aren't laid out as plain tensors. If your encoder is one of those, this node won't help - use CLIP Text Encode (Streamed LLM, from File) from the same pack, which streams the file itself and needs only the tokenizers from the CLIP input. The author is explicit that the file variant exists for exactly this case.

Wrong layer stack. The node streams the largest layer list it finds. On an exotic encoder with an unusual structure that heuristic could pick something unhelpful, and you'd see streaming "working" with no VRAM relief. Confirm from the console output rather than assuming.

Expecting a speedup. This is a memory node, not a speed node. If your encoder already fits, streaming it costs you transfers and buys nothing - leave the node bypassed, or trust auto_config to decide.

The Anima 40-block LoRA remap. Loading this pack installs a hook that lets 28-block Anima LoRAs apply to the 2.9B / 40-block variant. That's global and unrelated to text encoding, but it's the kind of install-time side effect worth knowing about before you blame a LoRA. WEIGHTFERRY_LORA_REMAP=0 turns it off.