Nodes/ComfyUI-ELLA/Combine CLIP & ELLA Embeds
ComfyUI Node

Combine CLIP & ELLA Embeds

Combine CLIP & ELLA Embeds — getting your LoRA trigger words back

By TencentQQGYLab·Created 2 years ago·Updated 2 years ago· 382
Combine CLIP & ELLA Embeds
  • cond
  • embeds
  • ELLA_EMBEDS

ELLA reads your prompt with FLAN-T5, and that's great for natural language. But here's the trap: your LoRA trigger words live in CLIP's vocabulary, and FLAN-T5 has no idea what they mean. Fire up a pure-ELLA workflow with a LoRA that needs shirtless guy or whatever token it was trained on, and the LoRA quietly does nothing. This node is the pack's fix.

Combine CLIP & ELLA Embeds takes a CONDITIONING (your normal CLIP conditioning from a CLIPTextEncode, which the LoRA has already leaked its trigger into) and an ELLA_EMBEDS bag, and merges them into one bag - it stashes the CLIP conditioning under an ella_clip_embeds key alongside the T5 embeddings, so the downstream apply/encode stage has both. If the bag already has a clip embeds entry, it warns you and overwrites it, so don't wire two CLIP sources in.

Inputs and output

  • cond - a CONDITIONING from the regular CLIP side of your graph.
  • embeds - the ELLA_EMBEDS bag from T5 Text Encode #ELLA.
  • Output - a merged ELLA_EMBEDS that carries both the T5 and the CLIP content.

Wire it in the middle of the modern chain:

CLIPTextEncode ─────────────┐
                            ├→ Combine CLIP & ELLA Embeds → ELLA Encode → KSampler
T5 Text Encode #ELLA ───────┘

The other route, and the README gotcha

There are actually two ways to get CLIP tokens back into an ELLA pipeline. This node bakes them into the embeds bag. The other way is to keep the CLIP conditioning separate and concatenate it with ELLA's CONDITIONING output using the Conditioning (Concat) node - and the README is emphatic about the wiring there: ELLA's conditioning always needs to go into the conditioning_to input of Conditioning (Concat), not the conditioning input. Get it backwards and the concat produces nonsense.

The simplest option of all is ELLA Text Encode, which does this CLIP concat automatically when you wire a clip in. But if you're composing your embeds explicitly - mixing CLIP content with T5 content for specific reasons, or chaining via ELLA Combine Embeds - this is the helper to reach for.

This is also the node the two deprecated helpers point at: ConcatConditionEllaEmbeds and ConditionToEllaEmbeds are both marked "Deprecated, CombineClip instead," and the former is literally the same code. Build new workflows with this one.

Install is the pack install - ComfyUI Manager or git clone https://github.com/TencentQQGYLab/ComfyUI-ELLA into custom_nodes, then pip install -r requirements.txt - plus the ELLA model in models/ella and FLAN-T5 XL in models/ella_encoder.

Categoryella/helper

Inputs (2)

NameTypeDefaultDescription
condCONDITIONING
embedsELLA_EMBEDS

Outputs (1)

NameTypeDescription
ELLA_EMBEDSELLA_EMBEDS