张量注意力融合

Two 3D tensors - [batch, seq_len, hidden_dim] each - go in, one fused tensor comes out. TensorAttentionFusion's recipe: stack the two sequences together along the token axis, run self-attention over the combined sequence, and return the result. Every token in both prompts can now attend to every other token in both prompts, and the output carries the whole mixed sequence in one tensor.

It's the closest thing in this pack to a "blend two embeddings into one" primitive, and if you've seen TensorCrossAttention you know the shape of the caveat coming.

The inputs are few:

• tensor1, tensor2 - the pair. Both must be 3D with matching batch size. Mismatched sequence lengths are allowed - the node warns you and fuses at the combined length (the output is seq1 + seq2 long). Mismatched feature dims get auto-aligned via the pack's TensorShapeAdapter (random projection - compatible, not meaningful).
• n_heads (default 8) - the hidden dim must divide evenly by it.
• temperature (default 1.0, range 0.1–10) - the one knob you'll actually play with. The attention output is divided by this value. Lower temperature sharpens the attention distribution (more concentrated on the strongest matches); higher spreads it out. It's a softness dial on the fusion.

Now the same asterisk that applies to the whole attention family in this pack: the multi-head attention layer is a randomly initialized nn.MultiheadAttention with no trained weights and no saved state. There is no learned notion of what should attend to what. So "fuse these two prompts" produces a mathematically consistent, shape-correct tensor whose content is a random projection-flavored mix. Deterministic, but not semantically meaningful on its own.

Where it's actually useful: as a scaffold. This pack saves embedding training pairs (SaveTrainingDataPair), and the tensor/conditioning family reads like building blocks for a custom training or experimentation pipeline. If you're building a graph you intend to train later - where these random attention layers are the architecture you'll learn weights into - this is a working, GPU-native, mask-handling primitive. If you hoped to merge a style prompt and a content prompt into something better, expect to be underwhelmed.

Install

ComfyUI Manager → search comfyui-spawner-nodes → install → restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/spawner1145/comfyui-spawner-nodes

Deps are three pure-Python libs (piexif, pypng, xmltodict) - no models, no torch extras. README is a one-liner, UI is Chinese-labeled, author (spawner1145) is the Chinese-speaking dev of a Wan2.1 SD extension. Works, barely documented.

Troubleshooting

• "必须是3维张量" - both inputs need [B, S, H]. 2D or 4D tensors are rejected up front.
• Batch mismatch - the two tensors must share B. Different seq or feature lengths are tolerated (with warnings); different batch is not.
• Embedding dim not divisible by heads - pick a n_heads that divides your hidden dim (4/8/16 on 768-dim).
• Fusion looks like noise - see the asterisk. Random weights. It's a training scaffold, not a finished mixer.