NNT Define Relative Position Bias

Absolute positional encodings tell a transformer "token 5 is here." But a lot of what a model needs to learn is relative - "token 5 is two steps after token 3," not "token 5." NNT Define Relative Position Bias is the node that adds that information: it injects a learned bias into the attention scores based on the distance between query and key positions, instead of adding position numbers to the embeddings. It's the mechanism that made Transformer-XL and a long line of efficient models work, and it's a small node with an outsize idea inside.

What it actually does

It appends a RelativePositionBias entry to the LAYER_STACK list that NntCompileModel compiles into a model. The trick is in the bucketing: instead of learning a separate bias for every possible distance (which explodes as sequences get long), distances get grouped into a fixed number of buckets - nearby distances each get their own bucket (precision where it matters), far distances get lumped together (because who cares exactly how far apart 500 and 510 are). num_buckets controls how many of those groups there are, and max_distance is where the "everything beyond here shares a bucket" cutoff kicks in.

Inputs that matter

• num_buckets - how many distance groups to learn biases for. 32 is the default; more buckets = more precision, more parameters.
• max_distance - the distance at which all farther pairs collapse into the same bucket. Keeps the bias table from growing with sequence length - this is the whole scalability trick.
• num_heads - attention heads; each head learns its own bias table.
• causal - True for autoregressive use, where keys never come after queries (the bias only needs one triangular half of the distance matrix).

How it fits together

This node pairs naturally with NntDefineTransformerXLAttention (which is the attention layer built around relative position) and NntDefineTransformerEncoderLayer. Think of it as the "distance awareness" component: where positional encoding colors the embeddings, this one colors the attention weights directly.

The honest gotcha

The pack-wide caveat applies with full force: NNT is a self-described work in progress, and the transformer corner is its most experimental. The node produces a clean, inspectable definition that's excellent for learning what relative-position bias is, but the current compile path doesn't guarantee every transformer entry materializes in the compiled model yet. Use NntCompileModel's "Only create script" mode to see what actually builds, and treat this as a teaching tool. Understanding bucketing by playing with num_buckets and max_distance is genuinely the best thing you can do with this node.

Installing NNT

Part of inventorado/ComfyUI_NNT. ComfyUI Manager (search "ComfyUI Neural Network Toolkit") or:

cd ComfyUI/custom_nodes
git clone https://github.com/inventorado/ComfyUI_NNT.git
cd ComfyUI_NNT
pip install -r requirements.txt

Restart ComfyUI after. Heavy dependency list - torch, numpy, scikit-learn, pandas, transformers, shap - so expect a long first install. The example workflows also want ComfyUI-Jjk-Nodes for text display; Manager's "Install Missing Custom Nodes" covers it.