Pinned PyTorch2.15, algorithm and introductory definition: parameter decay is a separate operation, while first/second moments are updated from the data gradient. The decay does not accumulate in those gradient moments. Used only to define this implementation convention; no library runtime executed.
Articles by others on the same topic
There are currently no matching articles.