VRAM Gated Diffusion Model Loader
Load FLUX (or Any Diffusion Model) Only After the VLM Steps Aside
- model
When your workflow runs a vision model to describe a video and then a diffusion model to generate from that description, the two models don't want to share a GPU. The VRAM Gated Diffusion Model Loader - the node's full display name - is the guard between them: it loads a UNET/diffusion model only after VidScribe MiniCPM has emitted its vram_cleared signal, meaning the VLM finished and got out of memory. The README calls it out specifically for FLUX and other UNET-based architectures, which is the modern case where checkpoints and diffusion models split apart.
Same honest caveat as the rest of the gated family: the gate is execution order, not VRAM telemetry. vram_signal is a required input, so ComfyUI simply cannot schedule this loader before the signal exists. It does not measure free memory, and if the model you pick genuinely doesn't fit your card, no signal will conjure headroom. What it kills is the co-residency problem - the VLM and the diffusion model both loaded at once.
Inputs that matter
- vram_signal - wire from VidScribe's
vram_cleared. - unet_name - dropdown from
ComfyUI/models/diffusion_models. - weight_dtype -
default, or fp8 variants:fp8_e4m3fn(the standard one),fp8_e4m3fn_fast(same dtype plus optimizations), andfp8_e5m2. On a tight card, fp8 is where you look first before touching the gating at all.
Output is a single model, ready for a KSampler or whatever your graph needs. Note there's no CLIP or VAE here - a diffusion model is just the UNET part, so if your pipeline needs text conditioning you're bringing CLIP/T5 from elsewhere (that's the point of the separate gated checkpoint loader).
How you'd use it
VidScribe (vram_cleared) ──► VRAMGatedUNETLoader (vram_signal) ──► model ──► KSampler
Caption first, generate second, no overlap. In a FLUX context where you're already fp8 and still tight, this is the node that keeps the captioning step from tipping a marginal setup over the edge.
Installing
Ships in TrentNodes under Trent/VLM. ComfyUI Manager → "Trent Nodes" → Install, or clone it:
cd ComfyUI/custom_nodes
git clone https://github.com/TrentHunter82/TrentNodes
cd TrentNodes
pip install -r requirements.txt
Restart after. If Manager flags the pack (a day-one repo rename left a registry duplicate it still distrusts), the manual clone is the reliable path - same as the rest of the pack.
Inputs (3)
| Name | Type | Default | Description |
|---|---|---|---|
| vram_signal | STRING | Connect vram_cleared from VidScribe | |
| unet_name | COMBO | 0 options: | |
| weight_dtype | COMBO | 4 options: default, fp8_e4m3fn, fp8_e4m3fn_fast, fp8_e5m2 |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| model | MODEL | — |