= Gradient descent preserves an eigenline for a cubic product
{title2=$h(x)=(x^TAx)(x^Tb),\quad x_n=t_nb\ \Longrightarrow\ t_{n+1}=t_n-3\alpha_n\lambda t_n^2$}
Let $A$ be real symmetric, let $b$ be a unit <eigenvector> satisfying $Ab=\lambda b$, and use scalar <step sizes> $\alpha_n$. The <gradient> of the displayed <cubic polynomial> <function> is
$$
\nabla h(x)=2(x^Tb)Ax+(x^TAx)b.
$$
On the <eigenline>, $\nabla h(tb)=3\lambda t^2b$, so a <gradient descent> update remains on that same line and obeys the scalar recurrence in the title. Induction proves invariance for every iterate starting on the line, including an iterate that reaches zero. This is a statement about the geometry of the updates, not a convergence theorem: when $\lambda\ne0$, the restriction $h(tb)=\lambda t^3$ is unbounded below in one direction.
Back to article page
