= Solution
Since $r^{(k)}=b-Ax^{(k)}=-\nabla f(x^{(k)})$, exact line search minimizes
$f(x^{(k)}+\alpha r^{(k)})$. Differentiating with respect to $\alpha$ gives
$$
-(r^{(k)})^Tr^{(k)}
+\alpha(r^{(k)})^TAr^{(k)}=0,
$$
so
$$
\boxed{\alpha^{(k)}
=\frac{(r^{(k)})^Tr^{(k)}}
{(r^{(k)})^TAr^{(k)}}}.
$$
Moreover,
$$
r^{(k+1)}=r^{(k)}-\alpha^{(k)}Ar^{(k)},
$$
and substitution of the displayed step size yields
$$
\boxed{(r^{(k)})^Tr^{(k+1)}=0}.
$$
\b[Thus consecutive residuals are orthogonal.]
Back to article page
