ComfyUI Node

Loader_SegmindVega

SDXL speed the pack's own way

By taabata·Created 3 years ago·Updated 2 years ago· 259
Loader_SegmindVega
    • class
    device
    tomesd_value0.6
    ip_adapter_model
    reference_only
    ip_adapter
    model_name

    Almost everything in this pack is SD 1.5 - the LCM Dreamshaper model is 1.5, the LCM-LoRA is the 1.5 one, all the pipelines are built around it. Loader_SegmindVega is the exception: it's an SDXL loader, built around Segmind Vega (the fast SDXL that was Segmind's answer to Turbo), sped up with its own dedicated LCM-style LoRA. It's the pack's "let's see if the same tricks work on SDXL" experiment, and the companion SegmindVega generate node is the only one in the pack that outputs a LATENT instead of an IMAGE.

    You'd reach for it if you want SDXL-class output at near-LCM speed and you're already inside this pack's ecosystem. It loads a Vega/SDXL model from models/diffusers/, fuses a Vega-specific LCM-LoRA (pytorch_lora_weights_vega.safetensors) and an LCMScheduler, and on GPU it adds VAE tiling and slicing on top of the usual xformers + CPU offload - memory tricks that matter because SDXL is heavy.

    How it works

    The load is StableDiffusionXLPipeline.from_pretrained(...) with a safety_checker=None override, then pipe.scheduler = LCMScheduler.from_config(...), then load_lora_weights + fuse_lora for the Vega LoRA. On GPU it enables xformers, sequential CPU offload, VAE tiling and VAE slicing - the tiling/slicing is what lets an SDXL pipeline survive on a 6–8GB card. On CPU it just runs fp32 and hopes.

    The reference_only and ip_adapter toggles are declared but mostly vestigial in the current code - the loader always builds the plain SDXL pipeline; the interesting conditioning choices live on the generate side.

    The inputs that matter

    • model_name - dropdown of diffusers folders in models/diffusers/. Where your Vega (or any SDXL) model lives.
    • device - GPU/CPU.
    • tomesd_value - ToMe ratio, 0.6 default.
    • ip_adapter / ip_adapter_model / reference_only - present for API parity with the other loaders; the generate node is where the real control happens.

    Output: class pipeline wire for the SegmindVega generate node.

    How to install it

    With the rest of the pack:

    cd ComfyUI/custom_nodes
    git clone https://github.com/taabata/LCM_Inpaint-Outpaint_Comfy
    cd LCM_Inpaint-Outpaint_Comfy
    pip install -r requirements.txt
    

    Or ComfyUI Manager → "LCM_Inpaint-Outpaint_Comfy" → restart. Then: a Segmind Vega SDXL model as a diffusers folder in models/diffusers/, and - the critical file - pytorch_lora_weights_vega.safetensors in models/loras/. Note the name: it's the Vega LoRA, not the 1.5 pytorch_lora_weights.safetensors the other loaders use. Get them mixed up and you'll get wrong-architecture errors or garbage.

    Common issues

    The file-name confusion above is the most common failure - the pack ships two different LoRA files with very similar names, one SD 1.5, one SDXL. If the loader throws an architecture mismatch, check which file is actually in models/loras/.

    Also be aware the current code's conditioning toggles are effectively cosmetic - if you flip ip_adapter to enable and expect the generate node to suddenly have IP-Adapter inputs, that's not how this one works. The generate node has its own ip_adapter toggle; keep them consistent and expect IP-Adapter behavior to be driven there.

    CategoryLCM_Nodes/nodes

    Inputs (6)

    NameTypeDefaultDescription
    deviceCOMBO2 options: GPU, CPU
    tomesd_valueFLOAT0.60–1
    ip_adapter_modelCOMBO0 options:
    reference_onlyCOMBO2 options: disable, enable
    ip_adapterCOMBO2 options: disable, enable
    model_nameCOMBO0 options:

    Outputs (1)

    NameTypeDescription
    classclass