Heavy-ball residual recurrence (source code)

= Heavy-ball residual recurrence

For the quadratic objective $F(x)=x^TAx/2-b^Tx$ with a real <symmetric matrix> $A$, the <Heavy-ball method> takes
$$
x_{k+1}=x_k+\alpha r_k+\beta(x_k-x_{k-1}),\qquad r_k=b-Ax_k.
$$
Multiplying by $-A$ and adding $b$ gives
$$
r_{k+1}=((1+\beta)I-\alpha A)r_k-\beta r_{k-1}.
$$
Under the usual zero-initial-momentum choice $x_{-1}=x_0$, one has $r_{-1}=r_0$, so induction expresses every residual as a <polynomial> in $A$ applied to $r_0$. It belongs to the associated <Krylov subspace>. With two arbitrary starting iterates, the general expression instead combines polynomials applied to both starting residuals; a single-residual polynomial claim then needs an additional relation between them.