ComfyUI Node

ELLA T5 Embeds

The 6GB Text Encoder That Replaces CLIP

By kijai·Created 2 years ago·Updated 2 years ago· 113
ELLA T5 Embeds
    • ella_embeds
    promptA vivid red book with a smooth, matte cover lies next to a glossy yellow vase. The vase, with a slightly curved silhouette, stands on a dark wood table with a noticeable grain pattern. The book appears slightly worn at the edges, suggesting frequent use, while the vase holds a fresh array of multicolored wildflowers.
    batch_size1
    max_length128
    fixed_negativefalse
    flexible_max_lengthtrue

    This node is the whole reason ELLA exists. SD 1.5's weak point was always its 77-token CLIP text encoder - you'd write a gorgeous five-element scene and get three of them, plus artifacts bleeding into the image. ELLA T5 Embeds swaps CLIP for a full flan-t5-xl encoder (T5-XL), and that's where the "semantic alignment" comes from: an LLM-sized text encoder that understands long, compositional prompts instead of mangling them. It's the direct fix for the prompt-adherence complaints that defined the SD 1.5 era.

    Mechanically, this node is a text encoder, full stop. You give it text, it runs the T5-XL encoder and returns an ELLAEMBEDS bundle - prompt_embeds, negative_prompt_embeds, and a batch size - that ELLA Sampler consumes. There is no CLIP here, no KSampler, no image. And there's no negative prompt text box either, which surprises people: the negative side is just an empty string, padded per the settings below. ELLA was trained against the empty string as the unconditional, so you mostly don't get to steer the negative.

    The inputs that matter:

    • prompt - multiline, and it has a long multi-clause demo default so you immediately see the point. Notice that T5 handles it without token bloat; max length is 512 tokens vs CLIP's 77.
    • max_length - default 128, up to 512. Only relevant when flexible_max_length is off.
    • flexible_max_length (optional, default on) - this is the setting that actually matters. ELLA's resampler is time-aware and wants each prompt at its natural token count, so when this is on, max_length is ignored and every prompt gets its true length. Leave it on.
    • fixed_negative - whether the empty-string negative gets padded to max_length (on) or kept at its natural length (off). Default off is correct.
    • batch_size - 1–256. This becomes the number of images the sampler generates.

    Output: ella_embeds (ELLAEMBEDS), wired into ELLA Sampler.

    The cost, and it's real: on first run this node downloads a ~6 GB flan-t5-xl encoder (Kijai/flan-t5-xl-encoder-only-bf16 into ComfyUI/models/t5_model/; older installs use the sharded ybelkada repo). That's a big download and then a few GB of VRAM while encoding - though it offloads back to CPU when done, which is why this pipeline stays livable on 8–12 GB cards.

    Install is the pack-standard route: ComfyUI Manager (search "ComfyUI-ELLA-wrapper"), or

    cd ComfyUI/custom_nodes
    git clone https://github.com/kijai/ComfyUI-ELLA-wrapper
    cd ComfyUI-ELLA-wrapper
    pip install -r requirements.txt
    

    On the portable Windows build, run python_embeded\python.exe -m pip install -r ComfyUI\custom_nodes\ComfyUI-ELLA-wrapper\requirements.txt from the portable root. Requirements are diffusers>=0.26.0, accelerate, omegaconf, sentencepiece - if your ComfyUI is behind, update diffusers first or the whole pack refuses to import.

    Troubleshooting: if you hit "layerNormKernelImpl" not implemented for 'Half' while this encodes - the signature ELLA-family error on fp16 systems - relaunch ComfyUI with --fp32-text-enc added to your launcher args. And remember: SD 1.5 only, and the pack is deprecated in favor of Tencent's official ComfyUI-ELLA. This one works, but it's a snapshot from mid-2024.

    CategoryELLA-Wrapper

    Inputs (5)

    NameTypeDefaultDescription
    promptSTRINGA vivid red book with a smooth, matte cover lies next to a glossy yellow vase. The vase, with a slightly curved silhouette, stands on a dark wood table with a noticeable grain pattern. The book appears slightly worn at the edges, suggesting frequent use, while the vase holds a fresh array of multicolored wildflowers.
    batch_sizeINT11–256
    max_lengthINT1281–512
    fixed_negativeBOOLEANfalse
    flexible_max_lengthoptBOOLEANtrue

    Outputs (1)

    NameTypeDescription
    ella_embedsELLAEMBEDS