Nodes/comfyui_gr85/Flux Attention Seeker 3
ComfyUI Node

Flux Attention Seeker 3

Per-block scaling on your CLIP text encoder

By veighnsche·Created 2 years ago·Updated 8 months ago· 1
Flux Attention Seeker 3
  • clip
  • CLIP
apply_to_querytrue
apply_to_keytrue
apply_to_valuetrue
apply_to_outtrue
clip_l_values_str1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
t5xxl_values_str1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0

A heads-up before anything else: "attention seeker" isn't an established community technique with a body of writeups behind it - it's this pack author's own experimental idea for poking at Flux's text encoding. That's not a knock on it, but set expectations accordingly: this is a tool for exploring what happens when you scale attention weights per layer, not a documented best-practice you're missing out on.

What it actually touches. Flux conditions on two text encoders - CLIP-L and T5-XXL - and this node intercepts the CLIP object between your loader and your text-encode node, applying a per-layer scaling factor to the query/key/value/output attention projections inside each. apply_to_query, apply_to_key, apply_to_value, and apply_to_out (all on by default) choose which of those four projection types get scaled - turn one off if you want to isolate the effect of, say, only the value projections. clip_l_values_str takes twelve comma-separated floats (CLIP-L has 12 transformer layers) and t5xxl_values_str takes twenty-four (T5-XXL's encoder stack). A value of 1.0 leaves that layer untouched; higher pushes that layer's attention output up, lower dampens it. The default for both is a flat string of 1.0s - a no-op baseline you're meant to edit.

Inputs and output. clip is a standard CLIP object, same as what comes out of your checkpoint or dual-CLIP loader. The four boolean toggles decide which attention components get touched. The two value strings are where the actual experiment lives - get the count wrong (more or fewer than 12 for CLIP-L, 24 for T5-XXL) and the mapping between numbers and layers won't line up the way you expect. Output is a single modified CLIP, which you wire into your CLIPTextEncode exactly like you would the unmodified one.

Installing it. Search "comfyui_gr85" in ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/veighnsche/comfyui_gr85

Restart ComfyUI. Nothing extra to download - it operates on a CLIP object you already have loaded.

Where you'll actually find the numbers to feed it. Typing twelve and twenty-four floats by hand isn't realistic for iteration, which is presumably why the same pack ships GR85_FluxAttentionSeekerGenerator (produces those comma-separated strings from a seed) and GR85_FluxAttentionSeeker2 (exposes the CLIP-L values as individual sliders instead of one string). If you're experimenting with this node, pair it with one of those rather than hand-editing the strings.

The honest caveat. Because this changes text-encoder attention rather than anything downstream, its effect on the final image is indirect and can be subtle-to-nonexistent depending on your prompt and CFG setup - don't be surprised if a moderate scaling change does very little. And since this is genuinely a personal exploration tool rather than a technique with known-good settings, there's no "use this value for realism" advice to give here; the only way to learn what it does is to set most layers to 1.0, change one at a time, and watch what shifts.

CategoryGR85/Prompt

Inputs (7)

NameTypeDefaultDescription
clipCLIP
apply_to_queryBOOLEANtrue
apply_to_keyBOOLEANtrue
apply_to_valueBOOLEANtrue
apply_to_outBOOLEANtrue
clip_l_values_strSTRING1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
t5xxl_values_strSTRING1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0

Outputs (1)

NameTypeDescription
CLIPCLIP