Pto AdamW

PtoAdamW wraps PyTorch's AdamW - the optimizer that modern Transformer training is built on. If you've ever wondered why every HF training script, every LoRA run, and this pack's own Transformer example uses AdamW instead of plain Adam, this node is the answer made clickable. It's the default choice for the pack's embedding_transformer_classification.json workflow, and it should probably be your default too.

Why AdamW and not Adam

Adam couples weight decay into the gradient updates, which sounds fine until you notice it fights with Adam's own per-parameter scaling - regularization gets dampened inconsistently. AdamW decouples weight decay: the weights get shrunk directly toward zero, separate from the gradient step. That small change is why AdamW trains Transformers and LLMs noticeably better than Adam, and why it became the de facto standard. For the text-classification Transformer in this pack, it's the difference between a model that converges cleanly and one that wanders.

How it works

The node calls torch.optim.AdamW(model.parameters(), lr, betas=(beta1, beta2), weight_decay, amsgrad). Inputs:

• model (PTMODEL) - the model whose parameters get optimized. Feed any model node's output here.
• learning_rate - default 0.001; Adam-family optimizers are usually happy there, tune down if loss explodes.
• beta1 / beta2 - gradient moment decays, defaults 0.9 / 0.999. Leave them.
• weight_decay - the L2-style regularization strength, applied directly to weights. Default 0.01 is the classic Transformer setting; bump it up when you're overfitting, drop toward 0 when your model underfits.
• amsgrad - the AMSGrad variant that keeps the running max of second moments. Off by default; it can help with tricky convergence but rarely needed.

Output is a single PTOPTIMIZER, which goes into a training node and optionally into an LR scheduler node (the Transformer example pairs it with cosine annealing).

The trap to avoid

Don't crank weight_decay just because you've heard regularization is good. For a small model on a small dataset, 0.01 is already meaningful; going to 0.1 can visibly cripple training. And if you're comparing results between this node and a friend's plain-Adam run, remember the difference you're seeing is partly AdamW's decoupled decay - that's a feature, not a bug. Also worth knowing: the optimizer binds to the model's parameters when created, so the model you wired in is the model you optimize - swap it after the fact and you're optimizing the old one. The pack's single-click trainer handles this for you.

Installing it

Part of ComfyUI-Pt-Wrapper (HowToSD's no-code PyTorch pack, a spin-off of ComfyUI-Data-Analysis). ComfyUI Manager → search "ComfyUI-Pt-Wrapper", or:

cd ComfyUI/custom_nodes
git clone https://github.com/HowToSD/ComfyUI-Pt-Wrapper

Restart after; first boot is slow while pandas, scikit-learn, transformers, sentencepiece, peft and friends install. No model files needed - the Transformer example fetches its dataset at runtime.