DiffSynth: VRAM Config

Big models don't fail to fit all at once. They fail in stages: while sitting unused, while being hauled onto the GPU, while being prepared for the forward pass, and during computation itself. That's the whole idea behind this node - four stages, each with its own device and its own numeric format. It's the first node in every DiffSynth workflow and the one people skip, then complain about the speed.

What it does

VRAM Config builds a dictionary that DiffSynth-Studio's loader understands, and you wire its vram_config output into the ModelConfig nodes. Each ModelConfig then carries that plan with it into the pipeline. It doesn't do anything on its own - an unwired VRAM Config is decoration.

The eight widgets, all enums:

| Stage | Device default | Dtype default | |---|---|---| | offload_device / offload_dtype | cpu | bfloat16 | | onload_device / onload_dtype | cpu | bfloat16 | | preparing_device / preparing_dtype | cuda | bfloat16 | | computation_device / computation_dtype | cuda | bfloat16 |

Devices are cpu, cuda, disk. Dtypes are float16, bfloat16, float32, float8_e4m3fn, and disk.

Read the defaults out loud and you have the whole mechanism: weights live in system RAM in bfloat16 while parked, get materialised on the CPU, and only move to the GPU for the parts of the job that need it - text encoding for preparing, forward passes for computation. That's why DiffSynth pipelines survive on cards that shouldn't manage them, and why they ping-pong over PCIe while doing it. GPU memory is roughly an order of magnitude faster than that bus, so every offload hop is a toll.

How to set it

• Card with headroom: push onload_device and preparing_device to cuda and leave computation there. Fewer hops, real speedup. If everything fits, this is free performance you're currently leaving on the table with the stock defaults.
• When something won't fit: leave onload/offload on cpu and let only computation sit on the GPU. On a 12GB card running a 14B video model, this is the difference between an error and a slow clip.
• disk in the device list means weights stay on disk and stream in. It's the last resort for a model that doesn't fit in RAM either; expect it to be genuinely slow rather than merely inconvenient.
• float8_e4m3fn is a real fp8 format, useful on Ada and newer. Whether it helps depends on the model and attention path, so A/B it rather than assuming.
• Mixed staging is the point of four separate slots: a small text encoder can sit entirely on cuda while a 14B transformer cycles through RAM.

Wiring it

VRAMConfig → vram_config  (of each ModelConfig)
                                   ↓
                        MergeModelConfigs → Loader → Inference

The VRAM Config feeds ModelConfig, not the Loader. This trips people up constantly. One VRAM Config can fan out to every ModelConfig in the graph; you don't need one per model, and if you do want different treatment per file - the SDXL case, with two text encoders and a UNet - that's exactly what separate ModelConfig nodes are for.

Install

Comes with the pack. Manager → DiffSynth-ComfyUI → install → restart, or git clone https://github.com/modelscope/DiffSynth-ComfyUI into custom_nodes after pip install git+https://github.com/modelscope/DiffSynth-Studio.git. Nothing here downloads models; it's pure configuration.

Gotchas

• Overridden without warning. The SD and SDXL loaders force their own torch_dtype onto the onload and computation stages. Set those two to something other than the Loader's dtype and you'll find out the hard way.
• Disk is not a typo. It's slow, and it's the only thing that works for a model larger than your RAM.
• Not a VRAM cap. This decides where things live; how much the pipeline may claim is a separate node, VRAM Limit.
• No effect without a ModelConfig. The settings travel with the individual model files, not with the pipeline.