= Residual and forward error in a stationary iteration
{title2=$r^{(k)}=-Ae^{(k)}$}
The <linear-system residual> $r^{(k)}=b-Ax^{(k)}$ is computable without knowing $x_*$. The <forward error> $e^{(k)}=x^{(k)}-x_*$ is the actual difference from the desired solution. For an <invertible matrix>,
$$
r^{(k)}=-Ae^{(k)},\qquad e^{(k)}=-A^{-1}r^{(k)}.
$$
Keeping $A$ fixed, the approximate <vector> exactly solves $Ax^{(k)}=b+\delta b$ with $\delta b=-r^{(k)}$. Thus, for $b\ne0$, $\|r^{(k)}\|/\|b\|$ is the relative <backward error> when only the right-hand side may change. For a <vector norm> and its induced <operator norm>,
$$
\frac{\|e^{(k)}\|}{\|x_*\|}\leq\kappa(A)\frac{\|r^{(k)}\|}{\|b\|},\qquad \kappa(A)=\|A\|\,\|A^{-1}\|.
$$
A small relative <linear-system residual> ensures a small relative <forward error> when the <condition number> is moderate; an ill-conditioned <matrix> can amplify it. The bound follows directly from $e=-A^{-1}r$ and $\|b\|=\|Ax_*\|\leq\|A\|\|x_*\|$.<Fundamentals of Numerical Computation by Tobin A. Driscoll and others/2.8 Conditioning of linear systems>[Fundamentals of Numerical Computation by Tobin A. Driscoll and others; §2.8.1, Theorem 2.8.1, equation (2.8.6), right-hand-side perturbation bound applied with Δb = −r]{ref}
For the <Jacobi iteration for a three-variable tridiagonal system>, the first <linear-system residual> is $(1,3,1)^T$, while the first <forward error> is $(-1/2,-1,-1/2)^T$. In the <Euclidean norm>, the relative <forward error> is $\sqrt{3/28}\approx0.3273$ and the relative <linear-system residual> is $\sqrt{11/120}\approx0.3028$. Its <spectral condition number of a positive-definite matrix> is $(4+\sqrt2)/(4-\sqrt2)\approx2.094$, so the bound is about $0.6340$. The bound is a guarantee, not an equality. In <floating-point arithmetic>, recompute the <linear-system residual> from the current <vector> when checking termination; a small update alone need not certify a small <forward error>.
Back to article page
