AdamW in the PyTorch documentation
= AdamW in the PyTorch documentation
{c}
https://docs.pytorch.org/docs/2.15/generated/torch.optim.AdamW.html
Pinned PyTorch2.15, algorithm and introductory definition: parameter decay is a separate $\theta\leftarrow\theta-\gamma\lambda\theta$ operation, while first/second moments are updated from the data gradient. The decay does not accumulate in those gradient moments. Used only to define this implementation convention; no library runtime executed.