张量交叉注意力

The raw-tensor cousin of ConditioningCrossAttention. Where that node works on CONDITIONING wires, this one takes three bare 3D tensors - query_tensor, key_tensor, value_tensor, each [batch, seq_len, hidden_dim] - and runs honest cross-attention: the queries attend to the keys, and pull the values. Out comes an output_tensor of the same shape as the queries.

The classic pattern you're re-implementing: make every token in one sequence aware of every token in another, so the output carries "query sequence, informed by key/value sequence." The same machinery sits at the heart of ControlNet-style conditioning and a thousand attention tricks, just applied here to raw tensors.

What you actually set:

• query_tensor, key_tensor, value_tensor - the three inputs. Key and value must have identical shapes (the node enforces it); query can differ in sequence length. Mismatched feature dims get auto-aligned with TensorShapeAdapter (random projection - dimension-compatible, semantically meaningless).
• n_heads (default 8) - hidden dim must divide evenly by it.
• add_residual (default on) - adds the original query back to the attention output (query + attention(query,key,value)). Residual on is the safe, standard choice; off gives you the raw attention readout.
• query_mask, key_mask (both optional) - the part that makes this node genuinely useful. These are 2D masks [batch, seq_len] in the pack's 1=keep/0=drop convention. key_mask becomes a proper key-padding mask, so masked key tokens genuinely cannot be attended to. query_mask is validated for shape but (per the source) isn't actually applied to the attention - so it's really there to keep you honest about shapes, not to filter queries. Build either mask with AllOnesMaskGenerator if you want "attend to everything" spelled out.

And the familiar asterisk: the attention layer is a stock nn.MultiheadAttention with random initialization and no learned weights. It's a deterministic, shape-correct primitive - a scaffold for training or experimentation, not a finished semantic fusion. If you feed it a "content" query and a "style" key and hope for a meaningful style transfer, you'll be disappointed. If you're building a graph you plan to train, this is a clean, working building block with the padding masks already wired up.

Install

Via ComfyUI Manager (search comfyui-spawner-nodes) or:

cd ComfyUI/custom_nodes
git clone https://github.com/spawner1145/comfyui-spawner-nodes

Restart after. Deps: piexif, pypng, xmltodict - light, no models. README is a stub, UI labels are Chinese, author is spawner1145 (of a Wan2.1 SD extension). The pack is a personal toolkit that escaped; useful, thinly documented.

Troubleshooting

• Key and value shape mismatch - enforced up front. Both must be [B, S, H] with identical shapes.
• Mask batch/seq mismatch - the node validates mask shape against its tensor and names the offender. Most common cause: mask built from a different tensor than the one you're feeding.
• "特征维度必须能被注意力头数整除" - adjust n_heads (4/8/16 for 768-dim embeddings).
• Query mask seems to do nothing - correct, per the source it's validated but not applied to attention. key_mask is the one that filters.