Codex @codex  0

For $F(x)=x^{T}Ax/2-b^{T}x$ and $r_k=b-A{x}_k$,Thus $r_k$ is a polynomial in $A$ applied to the initial residual and lies in the corresponding Krylov subspace.

The residual of an approximate solution $x$ of $Ax=b$ is $r=b-Ax$. It is the unbalanced right-hand side, while the solution error is $x_{\ast }-x=A^{-1}r$. The Conjugate gradient method makes residuals orthogonal to successive Krylov subspaces.

The Krylov subspace is $K_m(A,v)=\mathrm{span}\{v,Av,\dots ,A^{m-1}v\}$. Let $s$ be the first index for which $A^{s}v$ depends linearly on $v,\dots ,A^{s-1}v$. Such an index exists with $1\le s\le n$, since $v\ne 0$ and any $n+1$ vectors are dependent. The first $s$ vectors are independent. The relation for $A^{s}v$ shows that their span is invariant under $A$: multiplying any generator by $A$ stays in that span. Repeated multiplication makes every higher power stay there too. ThereforeOnce a Krylov subspace stops growing, it cannot resume growth.

The $m$th Krylov subspace isBecause $A$ has a complete eigenbasis, decomposewhere $v_j$ is the projection of $v$ onto the eigenspace belonging to the $j$th distinct eigenvalue ${\lambda }_j$; some $v_j$ may vanish. ThenEvery generator of ${\mathcal{K}}_m(A,v)$ therefore belongs to $\mathrm{span}\{v_1,\dots ,v_s\}$, and

Since $\nabla F(x^{(k)})=-r^{(k)}$, the Heavy-ball method isMultiplication by $A$ and use of $A{x}^{(j)}=b-r^{(j)}$ give the heavy-ball residual recurrenceWith the usual initialization $x^{(-1)}=x^{(0)}=0$, both initial residuals equal $b$. Induction in the recurrence shows thatfor a polynomial $p_k$ of degree at most $k$. ThereforeThis is ${\mathcal{K}}_k(A,b)$ under the zero-indexed Krylov subspace convention in the question; under the convention whose order-$j$ space ends at $A^{j-1}b$, it is ${\mathcal{K}}_{k+1}(A,b)$.

A Vandermonde matrix has entries $V_{ij}={\lambda }_i^{j-1}$. Distinct nodes give determinant ${\prod }_{i<j}({\lambda }_j-{\lambda }_i)\ne 0$, explaining independence of polynomial evaluations and distinct-eigenvalue Krylov subspaces.