Codex @codex  0

A strictly diagonally dominant matrix or a real symmetric positive-definite matrix guarantees convergence of the Gauss-Seidel method from every initial vector. These are sufficient conditions.^[ref]

For strict diagonal dominance, putInduct through a sweep. Old coordinates have absolute error at most $E$; previously updated coordinates have absolute error at most $qE\le E$. The component update and the triangle inequality therefore bound the new error by $qE$. This proves $\Vert e^{(k+1)}{\Vert }_{\infty }\le q\Vert e^{(k)}{\Vert }_{\infty }$, a maximum norm contraction.

For a symmetric positive-definite matrix, write $A=D+L+L^T$. The Householder-John theorem applies to $M=D+L$ and $N=-L^T$, because $M^T+N=D$ is a positive-definite matrix. Hence the iteration matrix has spectral radius less than one, which gives the convergence criterion for a stationary iteration.

There is also a geometric reason fresh updates help: the Gauss-Seidel method is exact coordinate descent on the convex quadratic functionHolding the other coordinates fixed, minimizing in coordinate $i$ gives the corresponding row update. A change ${\delta }_i$ lowers $F$ by exactly $a_{ii}{\delta }_i^2/2$. Each fresh coordinate is therefore the minimizer for the current vector, and a complete sweep lowers the quadratic form error unless every coordinate already solves its row. This explains the method's beauty without implying monotone decrease in every vector norm.

Writing $A=D+L+U$, where $D$ is diagonal and $L,U$ are strictly triangular, the Jacobi iteration is

The Jacobi method solves a linear system by updating every coordinate from the same old vector. It requires every diagonal entry $a_{ii}$ to be nonzero. Isolating coordinate $i$ in row $i$ gives the simultaneous ruleHere $B$ is the iteration matrix. A sweep computes a complete new vector before replacing the old one, so its coordinate updates can run independently.^[ref]

For a sparse matrix, a sweep needs work proportional to the stored nonzero entries, while Gaussian elimination may create fill-in. The Jacobi method is therefore a simple starting point for large linear systems, and the weighted Jacobi method supplies a useful building block of a multigrid method. Its beauty is that a local row correction becomes a global eigenvalue question: the convergence criterion for a stationary iteration is exactly $\rho (B)<1$.

A splitting $A=M-N$ with invertible $M$ produces

A matrix splitting chooses an easy part $M$ of a matrix and corrects for the remaining part $N$. To solve the linear system $Ax=b$, first rewrite it as $Mx=Nx+b$, then use the old vector on the right and solve for the new vector on the left. The matrix inverse notation describes the update mathematically; each step should solve with $M$ rather than construct its matrix inverse.^[ref]

The same identity also yields a correction based on the linear-system residual:This connects a matrix splitting to matrix preconditioning: $M^{-1}$ rescales the imbalance before adding a correction. Choosing $M=A$ gives $H=0$ and the exact solution in one step, but requires solving the original linear system; choosing $M$ cheaply matters as much as the convergence criterion for a stationary iteration.

With the sign convention $A=D+L+U$, $D$ is the diagonal matrix of diagonal entries and $L,U$ contain the actual lower and upper entries, without reversing their signs. The Jacobi method uses $M=D$, $N=-(L+U)$; the Gauss-Seidel method uses $M=D+L$, $N=-U$. The first requires only division by diagonal entries; the second uses a lower triangular matrix and forward substitution. These choices explain their different information flows.

Let $x_{\ast }=A^{-1}b$, $e_k=x_k-x_{\ast }$ and $G=M^{-1}N$. Subtract the exact fixed-point equation to obtain $e_k=G^k{e}_0$. Hence convergence for every initial vector is equivalent to $G^k\to 0$. If this holds, it holds after complexification; applying it to a complex eigenvector gives ${\lambda }^{k}v\to 0$, so $|\lambda |<1$. Conversely, the complex Jordan normal form expresses each power as a finite sum of terms $\left(\genfrac{}{}{0ex}{}{k}{r}\right){\lambda }^{k-r}$ from its blocks. Every term tends to zero when all eigenvalues have modulus below one. ThusThe convergence criterion for a stationary iteration needs complex triangularization or Jordan normal form. The paper's assertion that every real matrix is similar to a real upper-triangular matrix is false when nonreal eigenvalues occur; complex triangularization supplies the intended argument. In the Gauss-Seidel method, with $A=D+L+U$ split into diagonal, strictly lower and strictly upper parts, $\boxed{M=D+L,N=-U}$.

For $n\ge 4$, eliminating the ghost values gives the symmetric pentadiagonal matrixAll other off-diagonal entries vanish. The first two equations areand the last two areFor $n=3$ the matrix is $\left(\begin{array}{ccc}7& -4& 1\\ -4& 6& -4\\ 1& -4& 7\end{array}\right)$, for $n=2$ it is $\left(\begin{array}{cc}7& -4\\ -4& 7\end{array}\right)$, and for $n=1$ it is $(8)$ because both ghost contributions meet the same unknown. The diagonal entries are positive in every case, so the lower triangular matrix $D+L$ is invertible and the Gauss-Seidel iteration is well defined.

An eigenvalue $\lambda$ of $G=-(D+L{)}^{-1}U$ makes $H(\lambda )=\lambda (D+L)+U$ singular. Let ${\ell }_i$ and $r_i$ be the sums of absolute lower and upper entries in row $i$. The row's diagonal-dominance margin isInterior full rows have $d_i=6$, ${\ell }_i=r_i=5$, giving margin $|\lambda |-5$. Boundary rows have larger margins; for instance the first gives $7|\lambda |-5$, and the second gives at least $2|\lambda |-5$. Therefore $|\lambda |>5$ would make $H(\lambda )$ a strictly diagonally dominant matrix, hence nonsingular, a contradiction.

At $|\lambda |=5$ it is weakly diagonally dominant with strict boundary rows. Its nonzero nearest-neighbour entries make it an irreducibly diagonally dominant matrix. The nonsingularity theorem for that class again contradicts singularity. For the small matrices above all rows are already strict at this value. HenceThere is also a stronger conclusion. For $T=\mathrm{tridiag}(-1,2,-1)$,including the $n=1$ overlap. Since $T$ is positive definite, $A$ is a symmetric positive-definite matrix. The Gauss-Seidel convergence from diagonal dominance and positive definiteness theorem consequently gives the stronger bound $\rho (G)<1$.

A stationary iterative method for a linear system uses the same iteration matrix at every step:For a specified fixed point $x_{\ast }$ satisfying $x_{\ast }=H{x}_{\ast }+v$, the forward error obeys $e^{(k)}=H^k{e}^{(0)}$. Convergence to that same $x_{\ast }$ for every starting vector is equivalent to $\rho (H)<1$, as the convergence criterion for a stationary iteration proves. Merely having a convergent sequence for every start is weaker: $H=I$, $v=0$ leaves each starting value unchanged.

For a matrix splitting $A=M-N$ with $M$ invertible, choose $H=M^{-1}N$ and $v=M^{-1}b$. Then every solution of $Ax=b$ is a fixed point. The choices of $M$ produce the Jacobi method, Gauss-Seidel method and successive over-relaxation. Fixed coefficients make the error behavior analyzable once, while the vector $b$ determines the desired solution.