= Solution
Since $\nabla F(x^{(k)})=-r^{(k)}$, the <Heavy-ball method> is
$$
x^{(k+1)}
=x^{(k)}+\alpha r^{(k)}
+\beta(x^{(k)}-x^{(k-1)}).
$$
Multiplication by $A$ and use of $Ax^{(j)}=b-r^{(j)}$ give the <heavy-ball residual recurrence>
$$
\boxed{
r^{(k+1)}
=((1+\beta)I-\alpha A)r^{(k)}
-\beta r^{(k-1)}.}
$$
With the usual initialization $x^{(-1)}=x^{(0)}=0$, both initial residuals equal $b$. Induction in the recurrence shows that
$$
r^{(k)}=p_k(A)b
$$
for a polynomial $p_k$ of degree at most $k$. Therefore
$$
\boxed{
r^{(k)}\in
\operatorname{span}\{b,Ab,\ldots,A^kb\}.}
$$
This is $\mathcal K_k(A,b)$ under the zero-indexed <Krylov subspace> convention in the question; under the convention whose order-$j$ space ends at $A^{j-1}b$, it is $\mathcal K_{k+1}(A,b)$.
Back to article page
