SGDR: Stochastic Gradient Descent with Warm Restarts by Ilya Loshchilov and Frank Hutter (2017)

ICLR 2017, primary manuscript version 5 (3 May 2017). Full byline verified on page 1. Section 3, page 4, equation 5: cosine learning-rate schedule and retention of current parameters at restart. PDF arxiv.org/pdf/1608.03983v5 . Only optimization mechanism used, no biological application or universal benchmark guarantee.