Let be real symmetric, let be a unit eigenvector satisfying , and use scalar step sizes . The gradient of the displayed cubic polynomial function isOn the eigenline, , so a gradient descent update remains on that same line and obeys the scalar recurrence in the title. Induction proves invariance for every iterate starting on the line, including an iterate that reaches zero. This is a statement about the geometry of the updates, not a convergence theorem: when , the restriction is unbounded below in one direction.
Articles by others on the same topic
There are currently no matching articles.