Nodes/ComfyUI LLM SDXL Adapter/Apply T5Gemma LLM to Adapter
ComfyUI Node

Apply T5Gemma LLM to Adapter

T5Gemma's handoff to the sampler, size conditioning included

By NeuroSenko·Created about a year ago·Updated 10 months ago· 68
Apply T5Gemma LLM to Adapter
  • llm_hidden_states
  • llm_attention_mask
  • llm_adapter
  • CONDITIONING
width1024
height1024
target_width1024
target_height1024
crop_w0
crop_h0

t5gemmaApplyLLMToSDXLAdapter is the T5Gemma branch's counterpart to ApplyLLMToSDXLAdapter - the node that finally converts LLM output into a CONDITIONING tensor a KSampler will accept. The twist is that this one takes an attention mask as well as hidden states, and it hands the sampler the full SDXL size metadata alongside the conditioning. It's the last stop before sampling on the T5Gemma path:

T5GEMMALoader → T5GEMMATextEncoder → t5gemmaApplyLLMToSDXLAdapter → KSampler

How it works

The adapter forward pass is the same trained network from LLMToSDXLAdapter - projection to SDXL's 2048 sequence dim, wide attention, cross-attention compression down to the target sequence length, narrow refinement, then pooling to the 1280-dim vector. What differs from the Gemma apply node is what it does with the results:

  • It threads the attention_mask through the adapter's transformer blocks. Padding tokens get masked out properly instead of being treated as real content - the T5 encoder's padding="max_length" style makes that mask essential, and it's why this node demands it as an input.
  • It builds the conditioning metadata as {pooled_output, width, height, target_width, target_height, crop_w, crop_h} - the size and crop slots that SDXL conditioning carries around so the sampler knows the target geometry.

That metadata is the interesting part. SDXL's prompt conditioning normally includes the latent width/height/crop, and when you're generating at a specific size you want those to match your EmptyLatentImage. The optional width, height, target_width, target_height, crop_w, crop_h inputs (all defaulting to 1024/1024/0/0) let you set them directly on the node. For a normal 1024x1024 generation, leave them alone; they matter when you're generating at unusual resolutions and want the size conditioning to agree.

The inputs

  • llm_hidden_states (from T5GEMMATextEncoder) and llm_adapter (from LLMAdapterLoader, type: t5gemma - the 2304-dim preset, non-negotiable here).
  • llm_attention_mask - from T5GEMMATextEncoder. This node is the only one in the pack that uses it.
  • The six size/crop ints, all optional with defaults.

One output: CONDITIONING. No info string on this node, so debugging shape mismatches means reading errors instead of a nice text box.

Why this branch exists

The T5Gemma route trades the chat-template realism of the Gemma branch for a cleaner, mask-aware encoder pass - and it keeps SDXL's size conditioning honest. If you're training or running a T5Gemma adapter, this is the node that closes the loop. If you're not, the Gemma chain (LLMModelLoader → LLMTextEncoder → ApplyLLMToSDXLAdapter) is the documented, weights-available path and the one to start with.

Install

Ships with the ComfyUI LLM SDXL Adapter pack. ComfyUI Manager → "ComfyUI LLM SDXL Adapter", or git clone https://github.com/NeuroSenko/ComfyUI_LLM_SDXL_Adapter.git into ComfyUI/custom_nodes/, restart. Dependencies: transformers>=4.53.1, einops, safetensors, torch.

Troubleshooting

  • Shape mismatch on the adapter call - the classic type mismatch. Confirm LLMAdapterLoader is on t5gemma, not gemma.
  • Missing llm_attention_mask - you can't feed this node from the Gemma text encoder; it outputs no mask. Wire T5GEMMATextEncoder instead.
  • Odd geometry in results - check width/height here match the latent you're sampling into; SDXL reads both.
Categoryllm_sdxl

Inputs (9)

NameTypeDefaultDescription
llm_hidden_statesLLM_HIDDEN_STATES
llm_attention_maskLLM_ATTENTION_MASK
llm_adapterLLM_ADAPTER
widthoptINT102464–8192
heightoptINT102464–8192
target_widthoptINT102464–8192
target_heightoptINT102464–8192
crop_woptINT00–8192
crop_hoptINT00–8192

Outputs (1)

NameTypeDescription
CONDITIONINGCONDITIONING