Past exam of the year 1 of the computer science course of the University of Oxford / 2022 / Discrete mathematics, continuous mathematics and probability resit / 5 / c / iii

Use mathematical induction on the iteration. Initially $x_0=b=1\cdot b$. If $x_n={\lambda }_{n}b$, then $A{x}_n={\lambda }_{n}b/2$, $x_n^{T}b={\lambda }_n$, and $x_n^{T}A{x}_n={\lambda }_n^2/2$. ThereforeA gradient descent step with any scalar step length ${\alpha }_n$ consequently has the formThus every iterate remains on the same eigenline, withThis proves that gradient descent preserves an eigenline for a cubic product. The statement is invariance, not convergence to a minimum: along the negative direction of this line the cubic polynomial objective is unbounded below.