Ptn Embedding Transformer Linear

Ptn Embedding Transformer Linear is the pack's pre-built Transformer text classifier - embedding lookup, a full Transformer encoder stack, masked mean pooling, and a linear head, all inside one node. You configure it with the usual Transformer dials (layers, heads, hidden size, feedforward dim) and it produces a PTMODEL that turns token IDs into class scores. It's the node behind the pack's headline IMDB sentiment workflow, which the author says lands around 85% accuracy.

Why you'd reach for it

If your text problem has more than a couple of classes or a meaningful amount of data, Transformer encoders beat RNNs at the same parameter budget - they attend to every position at once instead of compressing the sentence through a hidden state. This node is the "don't build it yourself" path to that: one node gives you a real multi-head Transformer encoder. But it's also the perfect baseline for learning: crank num_layers from 6 down to 1, shrink nhead, and watch what each knob does to accuracy. Compared to Ptn Embedding RNN Linear, this is the more modern, usually stronger option; the RNN node is the simpler, cheaper one. The trade-off is training time - Transformers are hungrier.

How it works

Under the hood: an nn.Embedding maps token IDs to vectors, positional information is added, then a stack of nn.TransformerEncoderLayers (multi-head self-attention + feedforward, with your choice of GELU or ReLU activation and pre- or post-norm) processes the sequence. A masked mean-pooling step averages the token representations while ignoring padding, and a linear layer maps that pooled vector to linear_output_size scores. max_length sets the positional embedding size - it caps sequence length, and it's the one to raise if you're feeding long documents. Note the pack again defaults batch_first to True: input is (batch, seq, features).

The inputs

• vocabulary_size - vocab size; must cover your tokenizer's IDs.
• num_layers (default 6), nhead (8), hidden_size (512), dim_feedforward (2048) - the standard Transformer geometry. These four drive almost all of the model's cost.
• dropout (0.1), nonlinearity (gelu/relu), layer_norm_eps (1e-5), norm_first, bias - training and architecture details.
• max_length (512) - positional embedding length; sequence-length cap.
• linear_output_size / linear_bias - class count and head bias.

Output: PTMODEL.

Installing the pack

In the "Training" category. Install via ComfyUI Manager (search "ComfyUI-Pt-Wrapper") or:

cd ComfyUI/custom_nodes
git clone https://github.com/HowToSD/ComfyUI-Pt-Wrapper

Restart ComfyUI. Install pulls in transformers, datasets, peft, scikit-learn, gensim and more; the IMDB dataset downloads when you run the example workflow.

Common issues

• Hidden size vs nhead - hidden_size must be divisible by nhead (512/8 = 64 per head). A non-divisible combo throws at build time.
• Sequence too long - inputs longer than max_length get clipped or error. Raise max_length for long text, but remember it grows the positional table.
• Memory - the defaults are a real Transformer (6 layers, 512 hidden, 2048 feedforward). On a modest GPU that's slow to train; cut num_layers and hidden_size first.
• Thin community - single-author educational pack with almost no r/comfyui presence. The repo's training_transformer_for_classification and building_transformer_from_scratch docs are your best guides.