Heavy-ball rate for a two-eigenvalue diagonal quadratic (source code)

= Heavy-ball rate for a two-eigenvalue diagonal quadratic

Let $A=\operatorname{diag}(1,\gamma)$ with $\gamma>1$, choose $\alpha=1/\gamma$, and put $\rho=1-\gamma^{-1/2}$ and $\beta=\rho^2$. The two-by-two <heavy-ball error propagation matrix> for a scalar <eigenvalue> $\lambda$ has characteristic <polynomial>
$$
z^2-(1+\rho^2-\lambda/\gamma)z+\rho^2.
$$
At $\lambda=1$, its middle coefficient is $2\rho$, giving a repeated root $z=\rho$. At $\lambda=\gamma$, the polynomial is $z^2-\rho^2z+\rho^2$, whose roots are complex conjugates with modulus $\rho$. Thus the full propagation matrix has <spectral radius> exactly $1-\gamma^{-1/2}$.

The repeated root can produce an error term $(a+bk)\rho^k$, so the spectral radius is an asymptotic rate and does not assert a stepwise error bound with one fixed constant times $\rho^k$. For $\gamma=1$, the parameters reduce to $\alpha=1$, $\beta=0$ and the identity-matrix problem is solved in one update. Values $0<\gamma<1$ are outside the displayed rate convention.