Past exam of the numerical linear algebra course of the University of Oxford / 2011 / 2

The spectral theorem for real symmetric matrices gives $A=Q\Lambda Q^T$ with an orthogonal matrix $Q$. Positive definiteness makes every eigenvalue ${\lambda }_i$ positive. Define the matrix square rootIt is symmetric, its square is $A$, and for any nonzero vector $x$,Thus it is also positive-definite, and its matrix inverse is $A^{-1/2}=Q\mathrm{diag}({\lambda }_i^{-1/2})Q^T$.

The induced matrix norm associated with the given energy norm is a supremum of length ratios. The invertible change of variables $y=A^{1/2}x$ givesThe last expression is the Euclidean operator norm. This is the matrix norm induced by an energy norm: transforming to coordinates with Euclidean lengths transforms the operator by the corresponding matrix similarity.

Write $A=D+L+R$, where $D$ is the diagonal matrix of diagonal entries of $A$, and $L,R$ are its strictly lower- and upper-triangular parts. Since $A$ is symmetric, $R=L^T$. Subtract the two factors in the SOR iteration:Therefore, for $\omega \ne 0$,Positive definiteness makes each $D_{ii}>0$, so the triangular matrix $B(\omega )$ is an invertible matrix for every $\omega \ne 0$. The original formula gives $C(0)=I$; the expression for $B(\omega )$ itself is undefined there. The affine term ensuring that the desired solution is a fixed point is $c=B(\omega {)}^{-1}b$.

Apply the matrix norm induced by an energy norm and set $T=A^{1/2}C(\omega )A^{-1/2}=I-A^{1/2}{B}^{-1}{A}^{1/2}$. Multiplication givesThe bracket factors as $B^{-T}(B+B^T-A)B^{-1}$. Because $B=D/\omega +L$ and $A=D+L+L^T$,Thus $T^{T}T=M(\omega )$ for the stated matrix. In particular $M(\omega )$ is a positive semidefinite matrix. The Euclidean operator norm squared is the largest eigenvalue of $T^{T}T$, and all those eigenvalues are nonnegative. ConsequentlyThis identity is the algebraic step in the energy-norm proof of convergence of SOR; spectral radius equals the largest eigenvalue here because $M(\omega )=T^{T}T$.

For $0<\omega <2$, the coefficient $\gamma =2/\omega -1$ is positive. Moreoveris a positive-definite matrix: for a nonzero vector $z$, put $y=B^{-1}{A}^{1/2}z\ne 0$; then $z^{T}Gz=y^{T}Dy>0$. Hence every eigenvalue of $M=I-\gamma G$ is strictly below one. Part (d) also gives $M=T^{T}T$, so these eigenvalues are nonnegative. ThereforeThe energy-norm proof of convergence of SOR thus makes the iteration a strict contraction mapping in this norm. Its forward error satisfies $e^{(j+1)}=C(\omega )e^{(j)}$, so it converges geometrically to zero.