Positional Encoding

Here's a fact that surprises people: the Transformer has no built-in sense of word order. Its attention operation is a big weighted average - shuffle the input tokens and the math comes out identical. "The dog bit the man" and "the man bit the dog" would look the same to it. The fix is positional encoding, and ComfyDL's CdlPositionalEncoding is that fix as a node: it stamps each token's position into its embedding before attention ever runs.

How it works

The classic approach (from "Attention Is All You Need") uses sine and cosine waves, not learned parameters. For each position in the sequence, the node builds a fixed pattern: even feature dimensions get sin(position / 10000^(2i/d)), odd dimensions get the matching cosine. Because the frequencies differ per dimension, the pattern at position 5 is unique - and, nicely, nearby positions end up with related patterns, which is exactly what the model needs to reason about local structure.

The node precomputes this pattern once as a buffer of shape (1, max_len, num_hiddens). At forward time it takes your token embeddings X shaped (batch, seq_len, num_hiddens), adds the position slice for however many positions you actually used, and passes the result through a dropout layer. Nothing is trained; the encoding is baked in.

Inputs and outputs that matter

• num_hiddens (default 16) - your embedding/model width. Must match the last dimension of the embeddings you add it to. Even/odd dimensions get sin/cos respectively, so this is also the "how many frequency bands" knob.
• max_len (default 1000) - the longest sequence the precomputed pattern supports. Feed it a sequence longer than this and the addition will break, so bump it if you're experimenting with long inputs.
• dropout (default 0) - applied to the sum of embeddings plus position, right before output.

The model output is a cdlModel: give it token embeddings (not raw token indices) of shape (batch, seq_len, num_hiddens) and it returns the same shape with position baked in. In a hand-built pipeline it slots between the embedding layer and the first attention block. Wire it through a CdlModelForward to confirm shapes.

Installing ComfyDL

cd ComfyUI/custom_nodes
git clone https://github.com/Cynthia-lxx/ComfyDL ./ComfyDL
pip install -r ./ComfyDL/requirements.txt

Then restart ComfyUI. ComfyDL's requirements are featherweight (matplotlib, IPython, matplotlib-inline), and this node needs no model downloads. ComfyUI Manager users can search "ComfyDL", but this educational pack is young and may not be listed there - cloning is the dependable path.

Gotchas

Two easy mistakes. First, feed it embeddings, not token indices - if you connect a raw index tensor it will happily add a position pattern to garbage and return garbage-shaped-looking results. Second, respect max_len: it defaults to 1000, which sounds like plenty until you run a synthetic long-sequence experiment and watch the broadcast error fire. And a nice one to know: because the encoding is sinusoidal and not learned, you can inspect the P buffer directly to see how positions are encoded - a genuinely good way to build intuition.