Gradient descent preserves an eigenline for a cubic product

ID: gradient-descent-preserves-an-eigenline-for-a-cubic-product

Let be real symmetric, let be a unit eigenvector satisfying , and use scalar step sizes . The gradient of the displayed cubic polynomial function is
On the eigenline, , so a gradient descent update remains on that same line and obeys the scalar recurrence in the title. Induction proves invariance for every iterate starting on the line, including an iterate that reaches zero. This is a statement about the geometry of the updates, not a convergence theorem: when , the restriction is unbounded below in one direction.

New to topics? Read the docs here!