Heavy-ball residual recurrence Created 2026-09-24 Updated 2026-11-05
For the quadratic objective with a real symmetric matrix , the Heavy-ball method takes
Multiplying by and adding gives
Under the usual zero-initial-momentum choice , one has , so induction expresses every residual as a polynomial in applied to . It belongs to the associated Krylov subspace. With two arbitrary starting iterates, the general expression instead combines polynomials applied to both starting residuals; a single-residual polynomial claim then needs an additional relation between them.
Since , the Heavy-ball method is
Multiplication by and use of give the heavy-ball residual recurrence
With the usual initialization , both initial residuals equal . Induction in the recurrence shows that
for a polynomial of degree at most . Therefore
This is under the zero-indexed Krylov subspace convention in the question; under the convention whose order- space ends at , it is .