Nodes/TrentNodes/VRAM Gated Diffusion Model Loader
ComfyUI Node

VRAM Gated Diffusion Model Loader

Load FLUX (or Any Diffusion Model) Only After the VLM Steps Aside

By TrentHunter82·Created 9 months ago·Updated 4 days ago· 36
VRAM Gated Diffusion Model Loader
    • model
    vram_signal
    unet_name
    weight_dtype

    When your workflow runs a vision model to describe a video and then a diffusion model to generate from that description, the two models don't want to share a GPU. The VRAM Gated Diffusion Model Loader - the node's full display name - is the guard between them: it loads a UNET/diffusion model only after VidScribe MiniCPM has emitted its vram_cleared signal, meaning the VLM finished and got out of memory. The README calls it out specifically for FLUX and other UNET-based architectures, which is the modern case where checkpoints and diffusion models split apart.

    Same honest caveat as the rest of the gated family: the gate is execution order, not VRAM telemetry. vram_signal is a required input, so ComfyUI simply cannot schedule this loader before the signal exists. It does not measure free memory, and if the model you pick genuinely doesn't fit your card, no signal will conjure headroom. What it kills is the co-residency problem - the VLM and the diffusion model both loaded at once.

    Inputs that matter

    • vram_signal - wire from VidScribe's vram_cleared.
    • unet_name - dropdown from ComfyUI/models/diffusion_models.
    • weight_dtype - default, or fp8 variants: fp8_e4m3fn (the standard one), fp8_e4m3fn_fast (same dtype plus optimizations), and fp8_e5m2. On a tight card, fp8 is where you look first before touching the gating at all.

    Output is a single model, ready for a KSampler or whatever your graph needs. Note there's no CLIP or VAE here - a diffusion model is just the UNET part, so if your pipeline needs text conditioning you're bringing CLIP/T5 from elsewhere (that's the point of the separate gated checkpoint loader).

    How you'd use it

    VidScribe (vram_cleared) ──► VRAMGatedUNETLoader (vram_signal) ──► model ──► KSampler
    

    Caption first, generate second, no overlap. In a FLUX context where you're already fp8 and still tight, this is the node that keeps the captioning step from tipping a marginal setup over the edge.

    Installing

    Ships in TrentNodes under Trent/VLM. ComfyUI Manager → "Trent Nodes" → Install, or clone it:

    cd ComfyUI/custom_nodes
    git clone https://github.com/TrentHunter82/TrentNodes
    cd TrentNodes
    pip install -r requirements.txt
    

    Restart after. If Manager flags the pack (a day-one repo rename left a registry duplicate it still distrusts), the manual clone is the reliable path - same as the rest of the pack.

    CategoryTrent/VLM

    Inputs (3)

    NameTypeDefaultDescription
    vram_signalSTRINGConnect vram_cleared from VidScribe
    unet_nameCOMBO0 options:
    weight_dtypeCOMBO4 options: default, fp8_e4m3fn, fp8_e4m3fn_fast, fp8_e5m2

    Outputs (1)

    NameTypeDescription
    modelMODEL