AdamW in the PyTorch documentation

ID: adamw-in-the-pytorch-documentation

Pinned PyTorch2.15, algorithm and introductory definition: parameter decay is a separate operation, while first/second moments are updated from the data gradient. The decay does not accumulate in those gradient moments. Used only to define this implementation convention; no library runtime executed.

New to topics? Read the docs here!