Matrix inverse ($A^{-1}$)

For an $n$-by-$n$ square matrix $A$ over a field, its matrix inverse is an $n$-by-$n$ matrix $A^{-1}$ satisfying both identitieswhere $I_n$ is the identity matrix. It exists exactly when $A$ is an invertible matrix, equivalently when its matrix rank is $n$ or its determinant is nonzero. It reverses the linear map: applying $A$ and then $A^{-1}$ recovers the input, while applying $A^{-1}$ and then $A$ recovers the output.^[ref]

The matrix inverse is unique. If $B$ and $C$ both satisfy the identities, then $B=B(AC)=(BA)C=C$. Although the definition is two-sided, for finite square matrices either identity alone suffices: $BA=I$ forces a trivial kernel, whereas $AB=I$ forces surjectivity; either makes $A$ an invertible matrix by the rank-nullity theorem. Rectangular matrices may have only one of these properties.

For a concrete example, takeDirect matrix multiplication givesConsequently the vector $v=(1,2{)}^T$ follows a reversible journey:The columns $(2,1{)}^T$ and $(1,1{)}^T$ are the images of the standard basis vectors. They turn the unit square into a parallelogram; the matrix inverse restores both the square and its surrounding grid. Since $\det A=1$, this example changes shape while preserving area, illustrating that invertibility does not require preserving lengths or angles.

For comparison, the singular matrixcannot have a matrix inverse: the output $(1,0{)}^T$ cannot specify which of the two inputs occurred. Their difference $(1,-1{)}^T$ belongs to its kernel. Indeed $S(x,y{)}^T=(x+y,0{)}^T$ collapses the plane to a line, its matrix rank is one, and its determinant is zero. No subsequent function of the output alone can reconstruct every input.

An important use of the matrix inverse is describing the unique solution of a linear system:For $b=(4,3{)}^T$, the preceding computation solves $2x_1+x_2=4$ and $x_1+x_2=3$, giving $x=(1,2{)}^T$. In computation, usually solve the linear system using Gaussian elimination and a factorization such as an LU decomposition, rather than forming every entry of the matrix inverse. The inverse formula specifies the answer; a factorization provides a reusable way to compute it for each right-hand side.^[ref]

Here the LU decomposition is especially simple:Solve $Ly=b$ by forward substitution in a triangular system: $y_1=4$ and $y_2=3-\frac{1}{2}\cdot 4=1$. Then solve $Ux=y$ by backward substitution in a triangular system: $x_2=2$ and $x_1=(4-2)/2=1$. The same factors work when $b$ changes. If the whole matrix inverse is actually needed, its $j$th column is the solution of $Ax=e_j$, where $e_j$ is the $j$th standard basis vector.

Reversing several operations means undoing the last one first. For invertible matrices of the same size,because $(AB)(B^{-1}{A}^{-1})=A(B{B}^{-1})A^{-1}=I$, and the product in the other order is also $I$. This is why a change-of-basis matrix uses its matrix inverse to reverse the direction of coordinate conversion.

A claimed inverse matrix C for a finite square matrix A must satisfy $AC=CA=I$. A single product entry differing from the identity matrix disproves the claim. Conversely, in finite dimension one exact identity such as $AC=I$ already implies that A has full rank, so A is invertible and $C=A^{-1}$; the other product then equals I as well. Checking the product is useful before using a supplied inverse in a similarity transformation.

For $A=B=I$ of any positive size, $A+B=2I$ is invertible but $(A+B{)}^{-1}=I/2$, whereas $A^{-1}+B^{-1}=2I$. Thus even invertibility of all three matrices does not make inversion additive.

If two square matrices satisfy $AB=BA$ and $B$ is invertible, multiplication on the left and right by $B^{-1}$ gives $B^{-1}A=A{B}^{-1}$. Consequently a polynomial in a matrix commutes with the matrix inverse of any invertible polynomial in the same matrix.

A block matrix inverse expresses the matrix inverse in terms of smaller matrices. Partition a square matrix over a field asIf $A$ is an invertible matrix, define its Schur complement $S=D-C{A}^{-1}B$. Then $M$ is an invertible matrix exactly when $S$ is an invertible matrix, andThe order of the matrix products matters. In particular, the lower-right block is $S^{-1}$, rather than generally $D^{-1}$: the coupling through $B$ and $C$ changes the response.^[ref]

For an original example, takeThe Schur complement is $S=5-1/2-4/3=19/6$. Substitution gives the concise answerMultiplying in either order gives the identity matrix; the lower-right entry is $6/19$, whereas $D^{-1}=1/5$. The difference records the feedback through the two eliminated variables.

Often the desired output is a solution of a linear system, rather than every entry of a matrix inverse. Block elimination solves with $A$ and then with $S$, recovering the eliminated variables by back substitution. Reusing an LU decomposition of $A$ avoids explicitly forming $A^{-1}$. A favourable block diagonal matrix or tridiagonal matrix block can make this useful; merely partitioning an unstructured dense matrix does not guarantee a faster solve.^[ref]

Write a linear system as $Ax+By=f$ and $Cx+Dy=g$, with $A$ an invertible matrix. Solving the first equation for $x$ and substituting in the second givesThus the Schur complement is the coefficient matrix remaining after the $x$ variables have been eliminated. When it is an invertible matrix, solve the reduced linear system for $y$ and recover $x$. In computation, expressions such as $A^{-1}B$ mean solving with each column of $B$, using the same LU decomposition or Cholesky decomposition when applicable.

For the example under block matrix inverse and right-hand side $(3,1,4{)}^T$, take $f=(3,1{)}^T$ and $g=4$. Since $C{A}^{-1}f=13/6$, the reduced equation is $(19/6)y=11/6$. Back substitution givesThe original equations return $3$, $1$ and $4$, respectively. The reduction retained the influence of $x_1,x_2$ on $y$ through the correction $C{A}^{-1}B$.

A short proof outline also explains whether $M$ is an invertible matrix: the exact matrix factorizationhas invertible matrices on the outside. Therefore $M$ is an invertible matrix precisely when the central block diagonal matrix is one. Reversing the factors gives the block matrix inverse, while taking determinants gives the Schur complement determinant identity.

An invertible matrix need not have an invertible chosen diagonal block. For example,has singular matrices as both one-by-one diagonal blocks. Neither diagonal-block formula applies in that partition. Interchanging the two rows produces the identity matrix and a usable pivot element. This is why a failed block pivot is not proof that the full matrix is a singular matrix; Pivoted LU decomposition changes the order before eliminating.

Conversely, invertible diagonal blocks do not suffice: $M=\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)$ has $A=D=1$ but Schur complement $S=0$, so it is a singular matrix. Both hypotheses in the block matrix inverse formula have work to do.

The Sherman–Morrison formula computes the matrix inverse after a rank-one matrix update. If $A$ is an invertible matrix and $\delta =1+v^T{A}^{-1}u\ne 0$, thenThe inverse correction is itself an outer product, divided by the scalar $\delta$. The matrix determinant lemma shows that the nonzero-denominator condition is exactly the condition that the updated matrix remains an invertible matrix.

For $A=I$, $u=(1,2{)}^T$, and $v=(3,-1{)}^T$, we have $\delta =2$. Thus the computed inverse isMultiplication in either order gives the identity matrix. The result reuses the original inverse $I$ and the same correction vectors, rather than solving independently for all four inverse entries.

Often only the solution of an updated linear system is needed. If $A{x}_0=b$ and $Az=u$, thenprovided $1+v^{T}z\ne 0$. The two solves use the same LU decomposition of $A$. This expresses the computational benefit without requiring an explicitly stored matrix inverse. Strang's Lecture 14, transcript pp. 9–10 explains this reuse of the original factorization.

To see why the inverse correction works, first consider $A=I$ and put $s=v^{T}u$. The rank-one matrix $R=u{v}^T$ satisfies $R^2=sR$. Expandingalso works in the reverse order. The general case follows by factoring out $A$ and replacing $u$ by $A^{-1}u$. A matrix identity has reduced to cancellation of scalar coefficients.

Historically, Jack Sherman and Winifred J. Morrison studied updating an inverse when a single matrix entry changes in their ^[ref]. Such an entry change is the special rank-one matrix update $\alpha e_i{e}_j^T$, with standard basis vectors $e_i,e_j$. The reference and the earlier 1949 abstract on row or column changes are recorded in Henderson and Searle, On Deriving the Inverse of a Sum of Matrices (1981), references 40–41. This distinguishes the specific original setting from the more general $u{v}^T$ formula used here.

For an invertible matrix $A$, the rank-one matrix update $B=A+u{v}^T$ is a singular matrix exactly when $1+v^T{A}^{-1}u=0$. This follows directly from the matrix determinant lemma. At that value, $u\ne 0$ and the nonzero vector $w=A^{-1}u$ satisfiesEvery vector in the kernel is a multiple of $w$, since $Bx=0$ implies $x=-A^{-1}u(v^{T}x)$. The matrix rank therefore drops by exactly one.

For $B(t)=I+tu{v}^T$ with $u=(1,2{)}^T$ and $v=(3,-1{)}^T$, the update becomes a singular matrix at $t=-1$, and $B(t)u=(1+t)u$. At $t=-0.99$, this direction is compressed by a factor $0.01$ and the inverse amplifies it by a factor 100. The actual spectral norm of the matrix inverse, which maximizes amplification over every direction, is about $700.17853409$.

The exact formula can expose a large condition number in the updated linear system even when the original one had a small condition number. It does not guarantee numerical stability merely because its evaluation uses fewer arithmetic operations. The denominator condition certifies existence; the condition number and floating-point behavior address different questions about using the resulting answer.

Let $a,b,c\in {\mathbb{R}}^n$, with $c\ne 0$. The equation $x=a+(b\cdot x)c$ is the linear system $(I-c{b}^T)x=a$, a rank-one matrix update of the identity matrix. Every solution has $x=a+\lambda c$. Substitution reduces the entire problem toIf $b\cdot c\ne 1$, there is one solution, $x=a+c(b\cdot a)/(1-b\cdot c)$. If $b\cdot c=1$ and $b\cdot a=0$, every $\lambda \in \mathbb{R}$ works and the solution set is the straight line $a+\mathbb{R}c$. If $b\cdot c=1$ and $b\cdot a\ne 0$, there is no solution.

The first reduction follows directly from the original equation in any dimension. In three dimensions it can also be obtained by taking the cross product with $c$, giving $(x-a)\times c=0$. The singular denominator therefore represents two different possibilities, inconsistency or a family of solutions, and must not simply be discarded as an undefined formula.

The Moore-Penrose inverse extends the useful part of a matrix inverse to rectangular and singular matrices. A linear system $Ax=b$ may have no exact solution, or may have many. The pseudoinverse resolves both ambiguities: $x_{\ast }=A^{+}b$ is the minimum-norm least-squares solution. First minimize the Euclidean norm $\Vert Ax-b{\Vert }_2$; among all vectors attaining that minimum, choose the one with smallest Euclidean norm $\Vert x{\Vert }_2$. It does not pretend that inconsistent equations can all be satisfied.

For $A\in {\mathbb{C}}^{m\times n}$, its Moore-Penrose inverse is the unique $n\times m$ matrix satisfying the four Penrose equations. The singular value decomposition gives a direct construction:Here $U,V$ are unitary matrices, the star is the conjugate transpose, and ${\Sigma }^{+}$ is the $n\times m$ rectangular diagonal matrix obtained by replacing each nonzero singular value ${\sigma }_j$ by $1/{\sigma }_j$ and leaving zeros at zero. Over the real numbers, use orthogonal matrices and transposes. The pseudoinverse reverses the stretches that can be reversed; it discards output components the original linear map cannot produce and does not invent an input component in its kernel. ^[ref]

The two products have especially clear meanings:They are orthogonal projection matrices: $A{A}^{+}$ selects the producible part of an output, while $A^{+}A$ selects the detectable part of an input. They are generally not identity matrices. For an invertible matrix, both are identities and $A^{+}=A^{-1}$.

This gives a useful order of operations for least-squares solutions: project the requested output onto the column space, then invert the restriction of $A$ from $(\ker A{)}^{\perp }$ to its column space. Adding a kernel vector leaves the residual unchanged but increases the squared Euclidean norm by the Pythagorean theorem in an inner-product space, because $A^{+}b$ is orthogonal to that kernel. The pseudoinverse therefore provides one canonical answer without hiding either lost information or irreconcilable data.

The name records a history with several contributors. E. H. Moore gave an earlier generalization in 1920; M. H. Pearl's 1966 paper records that work, Arne Bjerhammar's 1951 rediscovery for maximal-rank rectangular matrices, and Roger Penrose's independent 1955 treatment. ^[ref] Penrose's original paper proves existence and uniqueness from the four Penrose equations for arbitrary complex matrices, including rectangular ones. ^[ref]

Suppose three measurements constrain two unknowns:The linear system asks for $x_1=1$, $x_2=2$, and $x_1+x_2=6$, which are incompatible. A least-squares solution makes a balanced compromise. The matrix has full column rank, soThe fitted output and residual areThus the two direct measurements are each overshot by one, and the measured sum is undershot by one. The residual has Euclidean norm $\sqrt{3}$ and is orthogonal to the column space, the plane $z=y_1+y_2$ in output coordinates $(y_1,y_2,z)$.

For any perturbation $h\in {\mathbb{R}}^2$, the orthogonality of $r$ to $Ah$ givesSince the kernel of $A$ is trivial, every nonzero $h$ increases the residual. The least-squares solution is unique in this example. The formula using $A^{T}A$ is convenient for these small exact calculations; an actual numerical computation can use a QR decomposition or singular value decomposition to avoid the extra conditioning cost of the least-squares normal equations. For full column rank, the squared singular values of $A$ are the eigenvalues of $A^{T}A$, so its condition number in the Euclidean norm is ${\kappa }_2(A^{T}A)={\kappa }_2(A{)}^2$. ^[ref]

Now take one equation for two unknowns:The linear system $x_1+2x_2=3$ has an entire affine line of exact solutions. Since $A$ has full row rank, its Moore-Penrose inverse isThe kernel is $\mathrm{span}\{(-2,1{)}^T\}$, and all exact solutions areBecause $x_{\ast }$ is orthogonal to $(-2,1{)}^T$,The Euclidean norm is therefore smallest exactly at $t=0$. The zero residual alone cannot select a solution; the minimum-norm rule supplies the additional criterion.

This selection depends on the specified inner product. Rescaling one unknown changes what counts as a small Euclidean norm, so units and weighting matter in an application. A minimum-norm answer is mathematically canonical for the chosen geometry, rather than an automatic guarantee of physical plausibility.

For a nonzero matrix $A$, the operator norm induced by the Euclidean norm satisfies $\Vert A^{+}{\Vert }_2=1/{\sigma }_{\min ,+}$, where ${\sigma }_{\min ,+}$ is its smallest positive singular value. The singular value decomposition makes the danger visible: a small nonzero stretch becomes a large amplification of errors when inverted.

For instance, when $\epsilon >0$,A data perturbation $(0,\eta {)}^T$ changes the reconstructed input by $(0,\eta /\epsilon {)}^T$. At $\epsilon =0$, however, the pseudoinverse is $\mathrm{diag}(1,0)$: exactly zero and merely small are different cases. Thus the pseudoinverse need not be continuous across a change of matrix rank. With uncertain data, truncated singular value decomposition or Tikhonov regularization can trade exact inversion of weak directions for stability; the cutoff or penalty adds a modelling choice.

For a bounded operator $A:X\to Y$ between Hilbert spaces, $A^{†}$ has domain $\mathcal{R}(A)\oplus \mathcal{R}(A{)}^{\perp }$, maps into $\mathcal{N}(A{)}^{\perp }$, and inverts the restriction of $A$ to $\mathcal{N}(A{)}^{\perp }$. It is bounded exactly when $\mathcal{R}(A)$ is closed.

The Moore--Penrose inverse is characterized by

Here $A^{†}=A^{+}$ denotes the Moore-Penrose inverse, while the star denotes the conjugate transpose. The first two identities express consistency when applying and reversing the linear map; the last two make the resulting orthogonal projection matrices Hermitian. They must hold together. For example, $A=(12)$ and $B=(10{)}^T$ satisfy $ABA=A$ and $BAB=B$, butis not Hermitian. This $B$ is a right inverse, yet $Bb=(3,0{)}^T$ is not the minimum-norm least-squares solution for $b=3$. The last two Penrose equations impose the orthogonality that selects the pseudoinverse.

The identity $(AB{)}^{†}=B^{†}{A}^{†}$ does not hold for arbitrary matrices. A sufficient condition is that $A$ have full column rank and $B$ have full row rank.

For a rectangular matrix $X$ and $\lambda >0$,Multiplying either side by $X{X}^T+\lambda I$ verifies the identity.