Transformer Encoder

The model class behind BERT and most modern text encoders, boiled down to a node that fits on your canvas. CdlTransformerEncoder builds a complete encoder stack in one shot: token embeddings, sinusoidal positional encoding, and a configurable number of stacked Transformer encoder blocks. Feed it token indices, get back a sequence of context-aware vectors. It's the "look, I can build one of those" node from ComfyDL's NLP Models category, and it's genuinely the best way to make the architecture stop being abstract.

The caveat that applies to all of ComfyDL applies here with extra force: this is a builder for understanding, not a pretrained model loader. You aren't getting BERT's weights. You're getting BERT's skeleton, freshly initialized, small enough to fit in RAM and inspect piece by piece. That's the entire value proposition - ComfyDL wraps the "Dive into Deep Learning" (d2l) textbook code into ComfyUI, and this is the transformer chapter's encoder, faithful to the text.

How the stack is put together

In order, the encoder does four things:

1. An nn.Embedding turns each token index into a num_hiddens-wide vector.
2. Those vectors are scaled by sqrt(num_hiddens) before anything else happens - a subtle textbook detail. The positional encoding that's about to be added lives in roughly [-1, 1], so without the rescale it would just get drowned out or dominate the learned embeddings.
3. A fixed sinusoidal positional encoding is added, so the model can tell position 3 from position 40 without any learned position parameters.
4. The sequence runs through num_blks identical encoder blocks (the same multi-head self-attention + add-norm + FFN + add-norm recipe as the pack's single-block node), and comes out shaped (batch_size, seq_len, num_hiddens) - the same shape it went in, now with context baked into every vector.

A genuinely nice detail for learners: after a forward pass, the encoder stashes each block's attention weights on the model (attention_weights), which is exactly what ComfyDL's Show Heatmaps node needs. That means you can run a tiny sequence through and see which tokens are attending to which - the classic BERT attention visualization, minus the mystery.

The inputs you'll actually set

• vocab_size - size of your token vocabulary; must match the vocab your token indices were built from.
• num_hiddens and num_heads - width and head count, and the one hard rule of the whole node: num_hiddens must be divisible by num_heads. The node refuses to build otherwise (it raises a ValueError before you even queue), which is friendlier than the cryptic shape error you'd otherwise get three layers deep.
• ffn_num_hiddens - the feed-forward bottleneck width, conventionally ~4× num_hiddens.
• num_blks - how many encoder blocks to stack. Default 2; the real BERT stacks 12–24, which is a great way to watch training time explode if you're curious.
• dropout and use_bias - regularization and attention-projection bias. Leave use_bias off (the default), matching modern practice.

Feed it token indices shaped (batch, seq) - the same indices Vocab Encode produces from text - and it returns a cdlModel you can push through Model Forward, inspect with Model Info, or chop open via Model Layers.

Single block or full encoder?

The pack also has Transformer Encoder Block, which builds just one. If your goal is to stack blocks by hand and watch each stage transform the sequence, use that one. If you want the whole architecture in a single wiring step - embedding, position, blocks and all - this is the node. For learning purposes, honestly, do the full encoder first to see the end state, then rebuild it from single blocks and confirm you get the same result. The d2l exercise, made graph-able.

Installing ComfyDL

Standard for every node in this pack:

cd ComfyUI/custom_nodes
git clone https://github.com/Cynthia-lxx/ComfyDL ./ComfyDL
pip install -r ./ComfyDL/requirements.txt

Restart ComfyUI and look under ComfyDL/NLP Models. The folder must be named exactly ComfyDL, and since the repo's registry PublisherId is still blank, ComfyUI Manager may not find the pack by name - the clone above is the reliable path. Requirements are just matplotlib, IPython and matplotlib-inline; torch and torchvision come with ComfyUI.

There's no community troubleshooting to cite for ComfyDL yet - the pack is new and niche. Your references are the repo's FUNCTIONS.md and the d2l transformer chapters, which this node exists to make legible.