ComfyUI-WeightFerry
Run diffusion transformers and LLM text encoders larger than your VRAM in ComfyUI by streaming their layers from RAM / SSD, bit-exact with ComfyUI; plus speed-ups for Anima (Cosmos-Predict2): fused block kernels, TaylorSeer, CFG early stop.
Nodes (17)
It ships, it works, and the author decided not to use it
Drop the negative branch for the last third and halve those steps' model calls
A drop-in encode node that can skip loading the text encoder entirely
The Anima encoder node for quantized text encoders
Windowed self-attention that measured slower, with seams
Fit a 14B diffusion model into 3GB of VRAM without quantizing it
A flow-matching sampler that decides for itself how many steps to skip
Redraw at 2x the trained size without tiling, if your model is Anima or Cosmos
Change the sampler in four places at once, on purpose
Skip steps by predicting them instead of computing them (Anima only)
When the T5 or Qwen encoder is the thing that won't fit
Reuse the prompt's cross-attention after step 6, and save almost nothing
A sigma table tuned for 10-step flow-matching sampling
The tiled upscaler, bundled, with a float16 upscale step and flat-tile skipping
The same 4x upscale, about 2.5x faster
Stop guessing tile sizes from your aspect ratio by eye
Tiled decoding without the seams, for the Wan 2.1 VAE
ComfyUI-WeightFerry
<img src="assets/logo.png" width="128" alt="WeightFerry logo: a ferry carrying model layers over waves" align="right">English | 日本語 — by nyanmi-1828
Run diffusion models that don't fit in your VRAM inside ComfyUI, with ComfyUI's own computation. For GPUs with 2–8 GB of VRAM, or any model larger than your VRAM. The model's blocks live in system RAM (or on an SSD) and are copied to the GPU just before they run, overlapped with the computation of the previous block.
Start with your goal
| Goal | First action | Node to search for |
|---|---|---|
| Reduce diffusion-model VRAM usage | Insert one node after the model loader / LoRAs | WeightFerry · Memory Saver (Auto) (WeightFerryMemoryAuto) |
| Use supported compute fusions | Install, restart and check supported conditions | No extra node for automatic fusions |
| Reduce text-encoder memory usage | Follow the CLIP instructions | Text Encoder Layer Streaming or CLIP Text Encode (Streamed LLM) |
| Set memory budgets / transfer options | Open advanced settings | DiT Layer Streaming (Advanced / 詳細) |
| Accept image differences to reduce computation | Read each feature's effect first | TaylorSeer / CFG Early-Stop (usage) |
Compute fusions are installed when the package loads and apply under their supported hardware, library, model and shape conditions. They do not require the memory node. The Auto node preserves the loader's precision and current attention; it does not convert bf16 to INT8. Its status reports memory management, not whether a compute fusion ran.
Notes
Loading the package enables TF32 (float32 operations differ slightly from strict float32) and registers supported
Anima / Cosmos-Predict2 block fusions and GGUF optimizations. ComfyUI standard on the simple node does not
remove these package-wide settings. Read global effects.
Requirements
- GPU: NVIDIA; tested on Ada (RTX 40 series, L4).
- ComfyUI: existing streaming nodes target 0.38.0 or later. The new Auto node / status events are developed against local 0.43.0; older frontend compatibility is not established by this statement.
- RAM: about the model's file size plus a few GB (Anima: 16 GB is enough); with less, see the SSD cache in Usage.
- OS: Windows 11 or Linux.
- Optional: SageAttention; Triton (
triton-windowson Windows) for the fused block kernels.
Install
This GitHub revision includes WeightFerryMemoryAuto. To use the new node, install manually below or update
your existing checkout from GitHub, then restart ComfyUI. Manager / Registry releases follow a separate update
schedule and may not include the new node yet.
Manual install:
cd ComfyUI/custom_nodes
git clone https://github.com/nyanmi-1828/ComfyUI-WeightFerry
Restart ComfyUI and reload its browser page. Search WeightFerry in the node menu. To uninstall, delete the folder and restart.
Minimal setup (low VRAM)
-
Load the diffusion model as usual (Load Diffusion Model / Load Checkpoint / Unet Loader (GGUF)), then the LoRAs.
-
Insert WeightFerry · Memory Saver (Auto) immediately before KSampler's MODEL input. Leave
memory_modeatAuto:Load Diffusion Model -> (LoRAs, if used) -> WeightFerry · Memory Saver (Auto) -> KSampler -
Keep CLIP, CONDITIONING, LATENT and VAE connections as usual. If the text encoder also needs less memory, use one separate CLIP streaming method from Usage.
-
Generate and read the node's memory status. Usage and Advanced open bundled help and a short advanced guide. The console records the node ID, run ID, configuration generation and reason. Console examples, the SSD cache and how each node is connected: docs/USAGE.md. Problems: docs/USAGE.md#troubleshooting.
The simple Anima example uses the official base model, a landscape prompt, ordinary CLIP Text Encode and KSampler. It adds no TaylorSeer, CFG Early-Stop or attention switch. Model files and settings. Existing larger examples retain their own settings.
Read the status
| Status | What is known | |---|---| | Not run / waiting | No result for this run | | Configured; waiting for model execution | Streaming registered; its execution has not been observed | | Streaming applied; model forward observed | A forward using streaming returned; full generation completion is shown separately | | Streaming not needed | The automatic planner chose no extra streaming; this is a configuration decision | | ComfyUI standard | This node's streaming is off; upstream and process-wide settings remain | | Upstream settings preserved | Another memory node already manages this MODEL connection | | Unsupported; standard path | See the CUDA / block-layout reason in the console | | Cached / cancelled / failed | Successful application is not inferred for this run |
Queue start, reconnect, setting / connection changes and workflow load clear the previous display. Cached models
can still report a newly observed streaming forward; a cached node alone does not prove a new execution.
ComfyUI standard does not force all weights onto the GPU or disable package-wide optimizations. Memory budgets
in advanced settings can affect ComfyUI model loads / VAE and are not restricted to one MODEL branch.
Output comparisons and troubleshooting
The model table below and results describe tested conditions, not every GPU / precision / feature combination. SageAttention, TF32, TaylorSeer, CFG Early-Stop and sampler / scheduler changes need separate comparisons. Activation streaming, VAE tiling and float16 upscaling each have their own documented numerical differences.
| Symptom | Next action |
|---|---|
| Node missing | Restart ComfyUI, reload the browser, then check revision / startup errors |
| Unsupported | Read the reason in the console and check tested models |
| No VRAM change | Streaming not needed is normal when the planner says the model fits |
| OOM | Identify MODEL / CLIP / VAE as the failing stage, then follow troubleshooting |
| First generation slower | Host weights are captured after loading; compare warm runs with equal settings |
Nodes
The full node table, advanced options and image effects are in Usage.
Measured examples
- Big models on a small GPU. Cosmos-Predict2 14B (fp8) runs on an 8 GB RTX 4060 in about 3 GB of VRAM instead of the 7 GB peak of ComfyUI's own run, at about the same speed; Qwen-Image 20B in 3.3 GB.
- Compared outputs. The tested weight-streaming paths have bit-for-bit comparisons against ComfyUI's own run of the same model. The exceptions: a full low-VRAM run with activation streaming differs slightly (LPIPS ~0.008), and the float16 upscale model inside Ultimate SD Upscale+ (its default) changes pixels slightly (LPIPS 0.012).
- Faster when VRAM is tight. With VRAM limited to 2 GB, Anima runs 1.65x faster than with ComfyUI's Dynamic VRAM; a GGUF Wan 2.1 14B at 4 GB runs 1.7x faster than with ComfyUI-GGUF alone (RTX 4060). Wan 2.1, Z-Image and Lumina 2 (Colab L4, 4 GB) run within ±7% of Dynamic VRAM.
The full feature list and all measurements: Detailed measurements.
Tested models
This table and the measurements record existing features. They do not establish that the new Auto node was retested across all models, precisions or GPUs.
For regression checks of the new Auto node, see tests/README.md. They use CPU dependency stubs and a mocked frontend; they are separate from GPU generation, ComfyUI API and real-browser validation of this published revision.
| Model | Tested | |---|---| | Anima (Cosmos-Predict2 2B based) | Existing nodes; VRAM 8 GB and limited to 4 / 2 GB | | Cosmos-Predict2 14B, Qwen-Image 20B | Streaming, bit-exact (RTX 4060 8 GB, 32 GB RAM) | | Wan 2.1 T2V 14B, Z-Image Turbo, Lumina Image 2.0 | Streaming, bit-exact (Colab L4, VRAM limited to 4 / 2 GB). Wan: images (1 frame) | | GGUF models (ComfyUI-GGUF): Wan 2.1 14B, Cosmos-Predict2 14B | Streaming, bit-exact, also with a LoRA | | Text encoders: Qwen3 0.6B, Qwen2.5-VL 7B, T5-XXL | Streaming, bit-exact |
Other DiTs (Flux, SD3, video) are untested. U-Net models (SDXL, SD 1.5) are not supported by streaming.
Documentation
- Usage and technical reference — all nodes, settings, measurements, global effects, troubleshooting and converters
- Environment variables — existing environment controls remain available
- Example workflows — model files and settings
- Third-party code and licenses
License
The package as a whole is distributed under the GNU GPL v3 (LICENSE), because it contains files adapted from GPL-3.0 projects. Inside it:
- Files adapted from other projects keep their licenses (GPL-3.0 from ComfyUI / TaylorSeer / loraPatch / Ultimate SD Upscale by ssitu and Coyote-A, Apache-2.0 from NVIDIA Cosmos / ComfyUI-GGUF; the ComfyUI-Anima-TeaCache parts, Apache-2.0, are inside the GPL-3.0 TaylorSeer file); each file's first line says which.
- This project's own code: MIT-0 — taken on its own, use it for anything, no conditions. A mention of this project is appreciated if you find it useful, but not required.
- The example workflows (examples/) and the logo (assets/): MIT-0 as well.
Details: THIRD_PARTY_NOTICES.md.
Status: a research project by one person.