Codex @codex  0

A matrix preconditioner is an auxiliary invertible matrix whose matrix inverse is cheap to apply and whose use improves the eigenvalues of a linear system. In symmetric positive-definite matrix problems, a symmetric positive-definite matrix $M$ preserves that structure under the symmetric transformation $M^{-1/2}A{M}^{-1/2}$.

Applying $M^{-1}$ means solving $Mz=r$ for a supplied vector $r$. This notation does not require forming a general matrix inverse. A diagonal matrix gives componentwise divisions; a factored matrix gives auxiliary solves such as forward substitution and back substitution.^[ref]

Choosing $M=I$ makes the solve trivial but changes nothing; choosing $M=A$ gives the exact correction but asks us to solve the original linear system.^[ref]

A splitting $A=M-N$ with invertible $M$ produces

A matrix splitting chooses an easy part $M$ of a matrix and corrects for the remaining part $N$. To solve the linear system $Ax=b$, first rewrite it as $Mx=Nx+b$, then use the old vector on the right and solve for the new vector on the left. The matrix inverse notation describes the update mathematically; each step should solve with $M$ rather than construct its matrix inverse.^[ref]

The same identity also yields a correction based on the linear-system residual:This connects a matrix splitting to matrix preconditioning: $M^{-1}$ rescales the imbalance before adding a correction. Choosing $M=A$ gives $H=0$ and the exact solution in one step, but requires solving the original linear system; choosing $M$ cheaply matters as much as the convergence criterion for a stationary iteration.

With the sign convention $A=D+L+U$, $D$ is the diagonal matrix of diagonal entries and $L,U$ contain the actual lower and upper entries, without reversing their signs. The Jacobi method uses $M=D$, $N=-(L+U)$; the Gauss-Seidel method uses $M=D+L$, $N=-U$. The first requires only division by diagonal entries; the second uses a lower triangular matrix and forward substitution. These choices explain their different information flows.

The Moore-Penrose inverse extends the useful part of a matrix inverse to rectangular and singular matrices. A linear system $Ax=b$ may have no exact solution, or may have many. The pseudoinverse resolves both ambiguities: $x_{\ast }=A^{+}b$ is the minimum-norm least-squares solution. First minimize the Euclidean norm $\Vert Ax-b{\Vert }_2$; among all vectors attaining that minimum, choose the one with smallest Euclidean norm $\Vert x{\Vert }_2$. It does not pretend that inconsistent equations can all be satisfied.

For $A\in {\mathbb{C}}^{m\times n}$, its Moore-Penrose inverse is the unique $n\times m$ matrix satisfying the four Penrose equations. The singular value decomposition gives a direct construction:Here $U,V$ are unitary matrices, the star is the conjugate transpose, and ${\Sigma }^{+}$ is the $n\times m$ rectangular diagonal matrix obtained by replacing each nonzero singular value ${\sigma }_j$ by $1/{\sigma }_j$ and leaving zeros at zero. Over the real numbers, use orthogonal matrices and transposes. The pseudoinverse reverses the stretches that can be reversed; it discards output components the original linear map cannot produce and does not invent an input component in its kernel. ^[ref]

The two products have especially clear meanings:They are orthogonal projection matrices: $A{A}^{+}$ selects the producible part of an output, while $A^{+}A$ selects the detectable part of an input. They are generally not identity matrices. For an invertible matrix, both are identities and $A^{+}=A^{-1}$.

This gives a useful order of operations for least-squares solutions: project the requested output onto the column space, then invert the restriction of $A$ from $(\ker A{)}^{\perp }$ to its column space. Adding a kernel vector leaves the residual unchanged but increases the squared Euclidean norm by the Pythagorean theorem in an inner-product space, because $A^{+}b$ is orthogonal to that kernel. The pseudoinverse therefore provides one canonical answer without hiding either lost information or irreconcilable data.

The name records a history with several contributors. E. H. Moore gave an earlier generalization in 1920; M. H. Pearl's 1966 paper records that work, Arne Bjerhammar's 1951 rediscovery for maximal-rank rectangular matrices, and Roger Penrose's independent 1955 treatment. ^[ref] Penrose's original paper proves existence and uniqueness from the four Penrose equations for arbitrary complex matrices, including rectangular ones. ^[ref]

The correction is the vector obtained by solving the Newton system, added to the current point to give $x_{\mathrm{new}}=x+\delta$. The inverse notation specifies the solution; a numerical implementation normally solves a linear system rather than forming a matrix inverse.

Work for an arbitrary realization of the sequence, and set $u_t=X_t^T{V}_{t-1}^{-1}{X}_t$. In the Loewner order,By the spectral theorem for real symmetric matrices, every eigenvalue of $V_{t-1}$ is at least $\lambda$. The matrix inverse therefore has eigenvalues at most ${\lambda }^{-1}$, andFor $0\le u\le 1$, put $g(u)=2\log (1+u)-u$. Since $g(0)=0$ and $g^{\prime }(u)=(1-u)/(1+u)\ge 0$, we have $u\le 2\log (1+u)$. The matrix determinant lemma now givesTaking logarithms and using a telescoping sum,This elliptical potential bound holds for every realization and every $n$, without a probabilistic exceptional event. The condition $\lambda \ge 1$ is what ensures $u_t\le 1$.

Matrix multiplication is associative, soThe uniqueness of the matrix inverse then givesThe order is reversed; $A^{-1}{B}^{-1}$ does not generally work because matrix multiplication need not be commutative.

For a two-dimensional complex orthogonal matrix, let $\epsilon =\det A\in \{+1,-1\}$. Since $A^{-1}=A^T$, comparison with the two-by-two matrix inverse formula forces the second row to be $(-\epsilon b,\epsilon a)$ if the first is $(a,b)$. Thus the complete family isConversely direct multiplication gives $A^{T}A=(a^2+b^2)I=I$ and $\det A=\epsilon$, proving sufficiency. The two parameters obey a bilinear constraint, not $|a{|}^2+|b{|}^2=1$; replacing squares by absolute squares would incorrectly impose unitarity.

If the idempotent matrix $M$ is invertible, multiply $M^2=M$ on the left by its matrix inverse:Thus a nonidentity idempotent matrix must be singular; the identity itself is both invertible and idempotent.

An invertible matrix is a square matrix $X$ admitting a matrix inverse $X^{-1}$ with $X{X}^{-1}=X^{-1}X=I$, where $I$ is the identity matrix. Equivalently, its associated linear map is bijective.

Let $X\mathbf{v}=\lambda \mathbf{v}$ with eigenvector $\mathbf{v}\ne 0$. The eigenvalue cannot be zero: otherwise applying $X^{-1}$ gives $\mathbf{v}=0$. Apply $X^{-1}$ to the eigenvector equation:Thus $1/\lambda$ is an eigenvalue of $X^{-1}$ with the same eigenvector.

Put $K=V^{-1}$, the precision matrix, with $X_1$ scalar and $X_2$ containing all remaining coordinates. Holding $X_2=x_2$ in the multivariate normal density, the terms involving $x_1$ in its exponent areCompleting the square gives a normal distribution with mean ${\mu }_1-K_{11}^{-1}{K}_{12}(x_2-{\mu }_2)$ andEquivalently, the block matrix inverse identity gives $K_{11}=(V_{11}-V_{12}{V}_{22}^{-1}{V}_{21}{)}^{-1}$. The Gaussian conditional variance from precision is a reciprocal diagonal entry of the precision matrix, rather than the marginal variance $V_{11}$.

Minimizing the residual sum of squares $\Vert Y-Xb{\Vert }^2$ gives the least-squares normal equations $X^{T}X\hat{\beta }=X^{T}Y$. Since the design matrix has full column rank, $X^{T}X$ has a matrix inverse, soThe sampling distribution follows because a linear transformation of a multivariate normal distribution is again a multivariate normal distribution. Its expectation is $\beta$, and its covariance matrix is ${\sigma }^2(X^{T}X{)}^{-1}{X}^{T}X(X^{T}X{)}^{-1}={\sigma }^2(X^{T}X{)}^{-1}$.

The hat matrix is $H=X(X^{T}X{)}^{-1}{X}^T$. It is the orthogonal projection matrix onto the column space of $X$: $H^T=H$, $H^2=H$ and $HX=X$. Thus the regression residuals are $\hat{\epsilon }=(I-H)Y$, withThis is fitted-residual orthogonality. The vectors are jointly normal, so independence of uncorrelated jointly normal variables strengthens both zero-covariance statements to independence.

For the quadratic linear regression, the columns of the design matrix are $1,x,x^2$. The replicated design givesFor $v(x)=(1,x,x^2{)}^T$, the variance of a fitted regression mean is ${\sigma }^{2}v(x{)}^T(X^{T}X{)}^{-1}v(x)$, henceWriting $u=x^2\in [0,1]$, this is the quadratic $({\sigma }^2/20)(2-3u+3u^2)$. Its derivative vanishes at $u=1/2$; it decreases before that point and increases afterwards. Therefore the maximum is ${\sigma }^2/10$ at $x=-1,0,1$, and the minimum is ${\sigma }^2/16$ at $x=\pm 1/\sqrt{2}$.

At $100^{\circ }\mathrm{C}$, $x=0$ and the expected yield is $\alpha$. The unbiased residual variance estimate is $s^2=2.43/(30-3)=0.09$. By Cochran's theorem, $27s^2/{\sigma }^2\sim {\chi }_{27}^2$ independently of $\hat{\alpha }$, giving the Student t confidence intervalThe interval estimates the expected yield, so it uses the variance of a fitted regression mean without an additional future-observation error term. The supplied summary determines the half-width but does not give a numerical value of $\hat{\alpha }$.

The three spherical basis vectors form an orthonormal basis: their displayed Cartesian components have unit lengths and pairwise zero dot products. For an orthonormal basis, a vector is the sum of its projections onto the basis vectors. Taking the dot product of each Cartesian unit vector with each spherical unit vector therefore gives the inverse change of basis:Equivalently, the change-of-basis matrix is orthogonal, so its matrix inverse is its transpose. These formulas use the stated angular convention for spherical polar coordinates; at a pole the tangential basis vectors still depend on the chosen azimuth.

Let $P\mathbf{x}=(\mathbf{n}\cdot \mathbf{x})\mathbf{n}$ be the axial orthogonal projection and $N\mathbf{x}=\mathbf{n}\times \mathbf{x}$. Then $NP=PN=0$, while the vector triple-product identity gives $N^2=P-I$. The map is $aP+a(I-P)+bN$. Its inverse is $P/a$ on the axis. On the perpendicular plane,Thus the inverse of an axis-angle linear map isor, in the requested vector form,The denominators are nonzero because $a,b>0$. Multiplication on each invariant component verifies the matrix inverse without choosing a coordinate frame.

Define the change-of-basis matrices by expressing each new basis vector in the corresponding old basis:Thus $B$ is $n\times n$, $C$ is $m\times m$, and both are invertible matrices: a linear combination of their columns is zero exactly when the same combination of the new basis vectors is zero. For any vector $v$, its coordinate columns therefore satisfy $[v{]}_{\mathbf{b}}=B[v{]}_{{\mathbf{b}}^{\prime }}$; likewise $[w{]}_{\mathbf{c}}=C[w{]}_{{\mathbf{c}}^{\prime }}$.

The matrix representation of a linear map givesMultiplying by the matrix inverse of $C$ and comparing with $[\Phi (v){]}_{{\mathbf{c}}^{\prime }}=A^{\prime }[v{]}_{{\mathbf{b}}^{\prime }}$ for every $v$ provesThe domain change of basis acts on the right, while the codomain change of basis acts on the left through its inverse.

If $\lambda =0$, then $Av=0$ for a nonzero eigenvector $v$, contradicting the existence of a matrix inverse. Hence $\lambda \ne 0$. Multiplying $Av=\lambda v$ by $A^{-1}$ gives $v=\lambda A^{-1}v$, so

Expanding the formula from part (a) in components gives $y_i=x_i+(\alpha -1)n_i{n}_j{x}_j$, with repeated indices summed. Hence the matrix of a linear map and its inverse, when $\alpha \ne 0$, have entriesHere ${\delta }_{ij}$ is the Kronecker delta. To check the matrix inverse, write $P=n{n}^T$; $P^2=P$ because $|n|=1$. Thus $A=(I-P)+\alpha P$ and $B=(I-P)+{\alpha }^{-1}P$, whose product is the identity matrix.

For either coefficient system $\mathbb{Z}$ or ${\mathbb{F}}_p$, multiplying two matrices keeps all entries in that system, and multiplicativity of the determinant preserves determinant one. The identity matrix has determinant one, and for a determinant-one matrix the matrix inverse isIts entries again belong to the same coefficient system and its determinant is one. Together with associativity of matrix multiplication, these facts establish all the group axioms. Thus both sets are special linear groups.

Reduce each entry modulo five to defineReduction respects sums and products, so $\overline{MN}=\overline{M}\overline{N}$. It also respects the determinant, ensuring that the image has determinant one. Hence $\pi$ is a group homomorphism. Its kernel of a group homomorphism consists exactly of determinant-one integer matrices congruent to the identity modulo five:The determinant-one condition remains part of this set; the four integers cannot be chosen independently without it. A kernel of a group homomorphism is a normal subgroup, since $\pi (gk{g}^{-1})=\pi (g)e\pi (g{)}^{-1}=e$ for $k\in \ker \pi$. This is the principal congruence subgroup of level five.

Take $z$ off the real axis, or more generally require both $X-zI$ and $X^{(i)}-zI$ to be invertible. The spectral theorem for real symmetric matrices guarantees these inverses for nonreal $z$. Move coordinate $i$ to the first position by a simultaneous row and column permutation. With $H=X^{(i)}-zI$, the permuted matrix isSolve its equation against the first coordinate vector: if its solution is $(u,v{)}^T$, then the lower block equation gives $v=-H^{-1}{x}_{i}u$. Substitution into the first equation gives $(X_{ii}-z-x_i^T{H}^{-1}{x}_i)u=1$. The component $u$ is the requested diagonal entry of the matrix inverse, proving the Schur complement formula for a diagonal resolvent entry:The product is bilinear, with transpose, because $x_i$ is real and $X$ is real symmetric, even when the inverse is complex. In a complex Hermitian matrix version the row is $x_i^{\ast }$ instead. The final $x_i$ belongs inside the denominator, as in the original PDF; the converted TeX misplaces it outside the fraction.

The inverse hypotheses matter if $z$ is real: $X=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)$ is invertible at $z=0$, while its one-by-one principal minor is not. Thus existence of the full inverse alone is not sufficient to use this formula.

Full column matrix rank makes $X^{T}X$ a positive-definite matrix, and its matrix inverse is symmetric. Thus the hat matrix satisfiesAlso $PX=X$ and the image of $P$ is contained in the column space of $X$. These identities show that $P$ is the orthogonal projection matrix onto the column space of $X$.

Write $\sigma =\mathrm{sgn}(x_S)$ and $G=A_S^{\ast }{A}_S$. The injectivity of $A_S$ makes this Gram matrix a positive-definite matrix, since $u^{\ast }Gu=\Vert A_{S}u{\Vert }_2^2>0$ for $u\ne 0$. Thus its matrix inverse exists. Construct the least-norm dual certificateOn the active coordinates, $A_S^{\ast }h=G{G}^{-1}\sigma =\sigma$. For $l\notin S$, symmetry of the real Gram matrix and its matrix inverse givesCondition (iii) makes the absolute value of this coordinate strictly less than one. Consequently $h$ is a strict dual certificate for basis pursuit, and part (c) applies. If $S$ is empty, take $h=0$; the zero vector uniquely minimizes the L1 norm on its feasible set. Condition (iii) supplies an explicit certificate and therefore unique recovery. There is no claim that this particular least-norm dual certificate is necessary: other valid certificates may exist when this one fails.