LLM Adapter Loader Unfused

If you've ever watched an SDXL model butcher a perfectly good natural-language prompt, the LLM-to-SDXL adapter scene has a seductive pitch: keep the SDXL UNet you love, but swap its CLIP text encoder for an actual LLM - Gemma or T5-Gemma - so the model finally reads your sentence instead of mashing tags into a bag. It works, sort of, through a small trainable adapter that translates LLM hidden states into the conditioning SDXL already understands. LLMAdapterLoaderUnfused is the part of that pipeline that loads the adapter - but only the variant trained with unfused QKV projections.

It ships in lRemixl/ComfyUI_LLM_SDXL_Adapter_Additions, a tiny add-on pack (four commits, one author, no issues) by Remix on top of NeuroSenko's ComfyUI_LLM_SDXL_Adapter. You don't run this pack on its own - it's the "plus" layer, and the upstream README is where all the real model downloads live.

What "unfused" actually means

Inside the adapter's transformer blocks, attention can be implemented two ways. The stock Rouwei-T5-Gemma adapter (Minthy/Rouwei-T5Gemma-adapter_v0.2) uses nn.MultiheadAttention, which fuses the query, key, and value weights into one big in_proj_weight tensor. The unfused variant keeps them as separate q_proj, k_proj, v_proj linear layers - a training choice that plays nicer with LoRA and some custom attention code. Same math, different weight layout.

Here's the gotcha that justifies this node's existence: a checkpoint trained one way doesn't drop into the other architecture's state_dict load. The stock fused adapter loads in the stock loader. An unfused-trained adapter needs a model with explicit q/k/v projections - which is exactly what this loader builds. The source even ships a convert_explicit_adapter_to_mha() helper to translate between the two formats, but it's not wired into the UI right now, so don't go hunting for a convert toggle.

The inputs that matter

• adapter_name - a dropdown that lists every .safetensors file it finds in ComfyUI/models/llm_adapters/ (it walks subfolders, so nested files show up with relative paths). This is the one you'll actually touch.
• type - gemma or t5gemma, default gemma. This picks the architecture preset: gemma uses an LLM dim of 1152 and 2 wide + 3 narrow blocks; t5gemma uses 2304 and 3 wide + 3 narrow. Match it to the adapter you loaded. Get it wrong and the checkpoint won't load, or loads and produces noise.
• device - auto / cuda:0 / cuda:1 / cpu. Leave it on auto unless you're juggling GPUs.
• force_reload - Boolean. The loader caches its adapter and only rebuilds when the name or type changes; flip this on if you replaced the file on disk and the node stubbornly keeps the old weights.

The two outputs are llm_adapter (type LLM_ADAPTER) and info (a STRING with the loaded path, type, device, and dims - handy to wire into a text preview while you're debugging). The llm_adapter output feeds straight into this pack's T5GEMMATextEncoder++ node's llm_adapter input, which is the encoder that runs your prompt through the LLM and the adapter and hands SDXL real conditioning.

Installing it

This is the add-on; the real setup is upstream's. Install this pack first:

cd ComfyUI/custom_nodes
git clone https://github.com/lRemixl/ComfyUI_LLM_SDXL_Adapter_Additions

Then restart ComfyUI. ComfyUI Manager also finds it if you search the pack title. You'll also want the upstream ComfyUI_LLM_SDXL_Adapter cloned the same way, plus the LLM (a T5-Gemma or Gemma model in ComfyUI/models/llm/ - you need all its files, not just the safetensors), the adapter in models/llm_adapters/, and your SDXL checkpoint. There's no requirements.txt in this pack; the real dependency is transformers >= 4.53.1 (that's the version where T5-Gemma support landed) plus einops and safetensors, which come from the upstream pack's requirements. The usual culprit when nothing loads: an old transformers.

Where people get burned

The number-one mistake is loading the stock fused Rouwei-T5-Gemma adapter into this unfused loader. The whole point of the node is the unfused variant, and the fused checkpoint's keys won't match the explicit-attention model - you'll get a load_state_dict error or silently wrong weights. If you're on the stock adapter, use the stock loader. If you're here, you presumably trained (or grabbed) an unfused checkpoint.

Second: pick the wrong type. The gemma vs t5gemma presets aren't cosmetic - llm_dim is 1152 versus 2304, and the checkpoint was built for one of them. Match it.

Third: an empty adapter_name dropdown. No .safetensors in models/llm_adapters/, no choices. Also remember the adapter is only the middle of the pipeline - no LLM in models/llm/ means the encoder has nothing to run, no matter how well this loader behaves.

Honest take: this is a niche corner of an already-niche technique. If you're not training your own unfused adapters, the standard loader is fine and this node adds nothing but confusion. If you are, it's the difference between your checkpoint loading correctly or fighting you.