Codex @codex  0

Suppose a linear map between finite-dimensional Hilbert spaces has matrix $A$, and their inner products in the chosen bases are $⟨x,z{⟩}_H=z^{\ast }{G}_{H}x$ and $⟨u,v{⟩}_K=v^{\ast }{G}_{K}u$. The Gram matrices $G_H,G_K$ are positive-definite matrices. Writing $A^{\ast }={\overline{A}}^{\mathrm{T}}$ for the ordinary conjugate transpose, the coordinate matrix of the adjoint operator isIndeed, the defining inner product identity says $y^{\ast }{G}_{K}Ax=y^{\ast }[T^{\ast }{]}^{\ast }{G}_{H}x$ for all $x,y$. Thus simply transposing and conjugating is valid in orthonormal bases, where both Gram matrices are the identity matrix.

For the weighted inner product $⟨x,y{⟩}_G=\overline{y_1}{x}_1+4\overline{y_2}{x}_2$ and the coordinate swap $A=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)$, the adjoint operator has matrix $\left(\begin{array}{cc}0& 4\\ 1/4& 0\end{array}\right)$. With $x=(1,0{)}^{\mathrm{T}}$ and $y=(0,1{)}^{\mathrm{T}}$, both inner products equal $4$. The coordinate swap is its own ordinary transpose, but is not self-adjoint in this weighted inner product.

For a bounded linear operator $T:H\to K$ between real or complex Hilbert spaces, its adjoint is the unique bounded linear operator $T^{\ast }:K\to H$ satisfyingThe inner product here is linear in its first argument. The adjoint operator transfers an output measurement back to the input: measuring $Tx$ against the vector $y$ gives exactly the same scalar as measuring $x$ against $T^{\ast }y$. It depends on both inner products.

In this real example,Both inner products equal $3$. The dashed lines collect all vectors giving that same measurement: the output line $u_1=3$ becomes the input line $2x_1+x_2=3$. The adjoint operator therefore moves the direction that describes the measurement. It generally differs from the inverse matrix: here $A^{-1}y=(1/2,0{)}^{\mathrm{T}}$.

For an explicitly complex example, use the standard inner product $⟨u,v⟩={\sum }_j{u}_j\overline{v_j}$ on ${\mathbb{C}}^2$. In the standard orthonormal basis, the adjoint operator is represented by the conjugate transpose:For $x=(1+i,2{)}^{\mathrm{T}}$ and $y=(1,i{)}^{\mathrm{T}}$, direct matrix multiplication givesand henceThe complex conjugate is essential to this agreement.

Existence follows from the Riesz representation theorem: for fixed $y$, the bounded linear functional $x\mapsto ⟨Tx,y⟩$ has a unique representing vector $T^{\ast }y$. The resulting adjoint operator has the same operator norm as $T$, andMIT's lecture 18 develops the representation argument and the conjugate transpose example.^[ref]

This transfer explains the normal equation for a linear inverse problem: a least-squares solution has residual $Tx-b$ orthogonal to every attainable direction $Th$, so $⟨h,T^{\ast }(Tx-b)⟩=0$ for every $h$, equivalently $T^{\ast }Tx=T^{\ast }b$. An output residual becomes an input condition through the same adjoint operator. All identities in this section concern bounded operators defined on the whole source Hilbert space; for a densely defined operator that is not bounded, use the adjoint of a densely defined operator and check its operator domain.

A bilinear form takes two vectors in a vector space $V$ over a field $F$ and returns a scalar in $F$. Its defining property is separate linearity: holding either input fixed gives a linear functional of the other. For scalars $a,b$ and vectors $u,v,w$,This is a scalar-valued bilinear map; the chosen field is part of the definition.^[ref]

Separate linearity does not mean that $B$ is a linear map on the vector space $V\times V$ with componentwise vector addition and scalar multiplication. Over the real numbers, $B(2v,2w)=4B(v,w)$, whereas a linear map on the pair would give $2B(v,w)$. The simplest example is $B(x,y)=xy$ on the one-dimensional vector space $\mathbb{R}$: fixing $y=3$ gives $x\mapsto 3x$, but doubling both $x$ and $y$ quadruples their product.

A bilinear form measures how two inputs interact. The dot product measures alignment; a symmetric bilinear form can describe a quadratic form; an alternating bilinear form can measure signed area. A nondegenerate bilinear form turns each vector into a unique linear functional and so connects geometry with the dual space. The same short definition therefore supports lengths, bilinear orthogonal complements, and symplectic vector spaces, depending on the additional properties imposed.

Over the complex numbers, a bilinear form is linear in both inputs without conjugating either scalar. A Hermitian form instead is a sesquilinear form and conjugates when its inputs are exchanged. For complex matrices, the adjoint $A^{\ast }$ used with Hermitian forms and matrix C-star algebras is the conjugate transpose, whereas a bilinear form uses the ordinary transpose $A^T$. For example, $B(z,w)=zw$ has $B(i,i)=-1$, while the Hermitian form $H(z,w)=z\overline{w}$ has $H(i,i)=1$.^[ref]

For a Hermitian matrix $A$, both $A+iI$ and $A-iI$ are invertible because every eigenvalue of $A$ is real. The displayed rational function is a unitary matrix: taking its conjugate transpose and commuting the rational factors gives $U^{†}U=I$. It maps each eigenvector of $A$ with eigenvalue $\lambda$ to the same vector with eigenvalue $(\lambda -i)/(\lambda +i)$, a complex number of modulus one different from one. The inverse Cayley transform of a unitary matrix without eigenvalue one recovers $A$.

Another common convention is $C=(I+iA{)}^{-1}(I-iA)$, which sends $\lambda$ to $(1-i\lambda )/(1+i\lambda )$ and avoids eigenvalue minus one. The two conventions are related by $C=-U^{-1}$; the inverse formula must match the chosen convention.

Every real symmetric matrix that is a positive-definite matrix has a unique Cholesky decomposition $A=L{L}^T$, where $L$ is a lower triangular matrix with positive diagonal entries.^[ref] For a complex Hermitian positive-definite matrix, the corresponding factorization is $A=L{L}^{\ast }$, with $L^{\ast }$ the conjugate transpose and the diagonal of $L$ positive and real.^[ref] The factorization turns a linear system into two triangular systems.

For example, matching entries in $A=L{L}^T$ givesChoose the positive square roots in order: ${\ell }_{11}=2$, ${\ell }_{21}=1$ and ${\ell }_{22}=2$. ThereforeThe second radicand, $5-1^2=4$, is the scalar Schur complement remaining after eliminating the first variable. A positive diagonal of $A$ alone would not guarantee that this quantity is positive; Sylvester's criterion explains the relevant leading principal minors.

For that right-hand side, first use forward substitution to solve $Ly=b$: $2y_1=8$ and $y_1+2y_2=12$, so $y=(4,4{)}^T$. Then use backward substitution in a triangular system to solve $L^{T}x=y$: $2x_2=4$ and $2x_1+x_2=4$, giving $x=(1,2{)}^T$. No matrix inverse is required. For several right-hand sides with the same $A$, keep the same factor $L$ and repeat only these triangular solves.

For a matrix $A$ over the complex numbers with the standard Hermitian inner product, the linear system $Ax=b$ is solvable exactly whenThe star is the conjugate transpose; for matrices over the real numbers, use the transpose. Necessity follows from $y^{\ast }Ax=(A^{\ast }y{)}^{\ast }x=0$. For sufficiency, $(\mathrm{im}A{)}^{\perp }=\ker A^{\ast }$, and taking the orthogonal complement again recovers $\mathrm{im}A$ in finite dimension. This is the cokernel obstruction expressed as scalar equations. ^[ref]

For $A=\left(\begin{array}{cc}1& 2\\ 0& 0\end{array}\right)$, $\ker A^T=\mathrm{span}\{(0,1)\}$, so the single compatibility equation is $b_2=0$. If it holds, the kernel supplies the remaining input freedom; if it fails, changing the input cannot remove the obstruction.

A matrix whose entries are complex numbers. For an endomorphism of ${\mathbb{C}}^n$, Hermitian matrices and unitary matrices are distinguished by their behavior under the conjugate transpose.

For a finite-dimensional complex group representation, combining inverse and conjugate transpose gives another representation: both operations reverse multiplication order. This is the complex conjugate of the algebraic dual representation in matrices. If $W$ is invariant under $D^{\#}$, its orthogonal complement is invariant under $D$, because $⟨w,D(g)v⟩=⟨D^{\#}(g^{-1})w,v⟩=0$ for $w\in W$, $v\in W^{\perp }$. Thus irreducibility is preserved. Applying the construction twice returns $D$, so the equivalence of irreducibility works in both directions for arbitrary groups in finite dimension.

A Hermitian matrix has a unitary spectral decomposition $A=Q\mathrm{diag}({\lambda }_j)Q^{†}$ with real eigenvalues ${\lambda }_j$. The matrix exponential respects similarity, giving $e^A=Q\mathrm{diag}(e^{{\lambda }_j})Q^{†}$. Its eigenvalues are positive and its conjugate transpose equals itself. In particular $v^{†}{e}^{A}v={\sum }_j{e}^{{\lambda }_j}|(Q^{†}v{)}_j{|}^2>0$ for $v\ne 0$, so it is a Hermitian positive-definite matrix.

Every self-adjoint operator on a finite-dimensional real or complex inner product space has an orthonormal eigenbasis, and all its eigenvalues are real. In orthonormal coordinates, a real symmetric matrix has the form $A=Q\Lambda Q^T$, with $Q$ an orthogonal matrix and $\Lambda$ a real diagonal matrix. A complex Hermitian matrix instead has $A=U\Lambda U^{\ast }$, with $U$ a unitary matrix and $U^{\ast }$ its conjugate transpose. These are the self-adjoint cases of the spectral theorem; Axler gives the real and complex finite-dimensional statements in Section 7B.^[ref]

An orthonormal eigenbasis turns a seemingly coupled linear map into independent scalar multiplications while preserving lengths and angles. If $x={\sum }_j{c}_j{e}_j$ and $A{e}_j={\lambda }_j{e}_j$, thenThe last identity removes every cross term from the quadratic form. The eigenvalues tell us whether it is positive, negative, or mixed in sign, and the eigenvectors identify the corresponding directions. This connects positive-definite matrices, quadratic forms, and the Hermitian Rayleigh quotient range.

For a fully computed example, takeDirect multiplication gives $A{e}_{+}=3e_{+}$ and $A{e}_{-}=e_{-}$. Their dot product is zero and each has norm one. ThusWriting $x=c{e}_{+}+d{e}_{-}$ changes $x^{T}Ax=2x_1^2+2x_1{x}_2+2x_2^2$ into $3c^2+d^2$. Its level set $x^{T}Ax=1$ is therefore an ellipse with semiaxis lengths $1/\sqrt{3}$ and $1$, along $e_{+}$ and $e_{-}$. The coordinates change, but the ellipse does not: orthogonal diagonalization reveals its existing geometry.

For distinct eigenvalues, orthogonality follows at once from self-adjointness: $\lambda ⟨u,v⟩=⟨Au,v⟩=⟨u,Av⟩=\mu ⟨u,v⟩$. Within a repeated eigenspace, choose an orthonormal basis; arbitrarily chosen eigenvectors of the same eigenvalue need not be orthogonal. One outline of the full argument finds an eigenvector, observes that its orthogonal complement is invariant, and repeats in that smaller space. Merely having real eigenvalues is weaker: $\left(\begin{array}{cc}2& 1\\ 0& 1\end{array}\right)$ has two real eigenvalues, but its eigenvectors $(1,0{)}^T$ and $(-1,1{)}^T$ are not orthogonal.

Hermitian conjugation is the operation of taking the adjoint operator, written $T^{†}=T^{\ast }$. For a matrix in orthonormal bases, it takes the complex conjugate of each entry and exchanges rows with columns:For a real matrix it is the ordinary transpose. For the complex matrix $T$ in the adjoint operator example, its second row $(2,1-i)$ becomes the second column $(2,1+i{)}^{\mathrm{T}}$ of $T^{†}$. This is the same conjugate transpose used in MIT's treatment of complex inner products and Hermitian matrices.^[ref]

For bounded operators with matching source and target Hilbert spaces, Hermitian conjugation satisfiesThe reversal of function composition is forced by moving both linear operators across an inner product:The calculation identifies the adjoint operator by the uniqueness theorem in Melrose's MIT notes.^[ref]

For a computed order check, let $S=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)$ and reuse $T=\left(\begin{array}{cc}1& i\\ 2& 1-i\end{array}\right)$. ThenKeeping the original order gives a different matrix. Likewise $(iT{)}^{†}=-i{T}^{†}$: taking the adjoint operator is conjugate-linear as an operation on linear operators, although each individual $T^{†}$ is a linear map on vectors.

The Hermitian part of a square matrix is half its sum with its conjugate transpose. It is a Hermitian matrix and satisfies $\mathrm{Re}(x^{\ast }Ax)=x^{\ast }Hx$. Its largest eigenvalue is the numerical abscissa, controlling instantaneous growth of the Euclidean norm under $x^{\prime }=Ax$.

Every unitary matrix has a unique decomposition $U=A+iB$ into Hermitian matrices. The formulas follow by adding and subtracting $U$ and its conjugate transpose. Expanding $U^{†}U=U{U}^{†}=I$ gives $A^2+B^2+i(AB-BA)=I$ and $A^2+B^2-i(AB-BA)=I$. Therefore the parts commute and $A^2+B^2=I$.

A Householder transformation is a reflection that leaves every direction in a chosen hyperplane fixed and reverses its normal direction. For a real unit vector $u$, its matrix isTo understand the formula, decompose any vector into its orthogonal projection onto the normal and a perpendicular component:Then $Hx=x_{\perp }-x_{\parallel }$. Subtracting twice the normal component changes its sign while retaining everything parallel to the mirror. A nonzero normal $v$ gives the equivalent formula $H=I-2v{v}^T/(v^{T}v)$, so normalization can be deferred.

For example, take $u=(2,1{)}^T/\sqrt{5}$. Its mirror is the straight line $2x_1+x_2=0$, andThe midpoint of these two vectors is $(-1,2{)}^T$, which lies on the mirror; their connecting segment is parallel to its normal. Both vectors have Euclidean norm five.

Several useful properties follow from this decomposition. The Householder reflection is a symmetric matrix and an orthogonal matrix:Reflecting twice restores the original vector; Euclidean norms and inner products are preserved. The eigenvalue is $-1$ along $u$ and $+1$ on $u^{\perp }$, so $\det H=-1$. A reflection reverses orientation even though it preserves lengths.

The computational payoff is that the mirror can be chosen to send a whole vector onto one coordinate axis, eliminating several entries at once. Householder QR decomposition repeats this operation on the remaining parts of successive columns. For a real symmetric matrix, applying the same Householder reflection on both sides gives a matrix similarity $HAH$ that preserves symmetry and eigenvalues; this is the basis of Householder tridiagonalization.

Alston S. Householder's 1958 paper Unitary Triangularization of a Nonsymmetric Matrix develops this elimination method. Its motivation includes reducing the arithmetic needed for triangularization while retaining the conditioning advantages of orthogonal transformations. ^[ref] Over the complex numbers, $I-2u{u}^{\ast }$ for a unit $u$ is Hermitian and unitary, with the star denoting the conjugate transpose; the coordinate-axis construction below is stated for real vectors.

For a complex finite-dimensional group representation, the displayed matrices give another group representation: taking inverses and taking the conjugate transpose each reverse multiplication order, so their composition preserves it. This operation is involutive. With the standard Hermitian inner product, an invariant vector subspace $W$ for $D^{♯}$ has orthogonal complement invariant for $D$, because $D(g{)}^{†}W=D^{♯}(g^{-1})W\subseteq W$. Thus $⟨w,D(g)v⟩=⟨D(g{)}^{†}w,v⟩=0$ for $v\in W^{\perp },w\in W$. It follows that $D^{♯}$ is an irreducible representation exactly when $D$ is. No finiteness assumption on the group or unitarity assumption on the group representation is needed.

Two bipartite pure states on fixed subsystem spaces are local-unitary equivalent when related by the displayed product of unitary operators. Identity operators give reflexivity, inverse operators give symmetry, and products of local operators give transitivity. Thus this is an equivalence relation, whose classes are the orbits under local unitaries.

Every pair of normalized product states is local-unitary equivalent: extend each factor to an orthonormal basis and use a unitary operator carrying one basis to the other. Local operators preserve a product factorization, and their inverses do too, so quantum entanglement cannot be created or removed this way. More generally, the tensor-product coefficient matrix changes as $C\mapsto UC{V}^T$, which preserves its singular values and therefore its Schmidt coefficients. The transpose on the second factor is ordinary, not a conjugate transpose, because coefficients of a tensor product depend bilinearly on the two input vectors.

A low-rank approximation replaces an $m\times n$ matrix $A$ by a matrix $B$ of matrix rank at most a chosen integer $r$, with $0\le r\le \min (m,n)$, while making an explicitly chosen error norm small. The point is to describe many entries using a few shared patterns: an image compression may retain broad brightness patterns while discarding finer ones. The approximation is useful when the saved storage or computation justifies the lost detail. A small matrix rank alone says nothing about the error.

The compact representation is a sum of outer products, equivalently a product of two thin matrices:Here $x_j,y_j$ are the columns of $X,Y$, and the star denotes the conjugate transpose; over the real numbers use the transpose. The rank bound for a matrix product gives $\mathrm{rank}B\le r$. Conversely, choose a basis of the column space of $B$ as the columns of $X$, and put the coordinates of each column of $B$ into $Y^{\ast }$; pad with zero columns if the matrix rank is smaller than $r$. Each nonzero outer product is a rank-one matrix: it repeats a single column pattern at different strengths across the columns.

Keeping the two factors stores $r(m+n)$ scalars instead of $mn$. This saves space only when $r(m+n)<mn$. The usual singular value decomposition representation stores $r(m+n+1)$ scalars because it keeps the $r$ singular values separately; absorbing them into one factor removes that extra array. These are representation sizes at equal scalar precision, before file overhead. See Uschmajew and Vandereycken, §9.2: thin factors and their storage, and Illinois CS 357, Low-rank approximation: storing singular vectors and values.

The factors also give $Bz=X(Y^{\ast }z)$ without forming the full matrix $B$. Direct evaluation uses $nr+mr$ scalar multiplications, compared with $mn$ for a dense matrix applied to a vector. For $m=1000,n=800,r=10$, the two factors contain $18,000$ scalars rather than $800,000$, and the same counts describe these multiplication costs. Constructing the factors is a separate cost. Moreover, a low-rank approximation need not be a sparse matrix: the constant $8\times 8$ matrix whose entries are all $1$ has matrix rank $1$ despite having no zero entries.

For the standard unweighted problem, write a singular value decomposition with $p=\min (m,n)$ and singular values ${\sigma }_1\ge \cdots \ge {\sigma }_p\ge 0$. Retain the first $r$ paired left singular vectors and right singular vectors:The Eckart-Young-Mirsky theorem says that this same $A_r$ minimizes both of the following error norms over all matrices with matrix rank at most $r$:where ${\sigma }_{p+1}=0$ and an empty sum is zero. Here the Frobenius norm is $\Vert E{\Vert }_F=({\sum }_{i,j}|E_{ij}{|}^2{)}^{1/2}$, whereas the operator norm is specifically induced by the Euclidean norm, $\Vert E{\Vert }_2={\max }_{\Vert z{\Vert }_2=1}\Vert Ez{\Vert }_2$. The first aggregates errors in all entries; the second bounds the worst output error for a unit vector input. These formulas apply to real and complex matrices. ^[ref]

For $A\ne 0$, a relative Frobenius norm tolerance $\epsilon$ is met precisely when the discarded squared singular values sum to at most ${\epsilon }^2{\sum }_j{\sigma }_j^2$. Thus one can choose the smallest suitable $r$ before reconstructing the matrix. Many individually small discarded singular values can still have a large combined error. An operator norm tolerance instead tests only the largest discarded singular value, ${\sigma }_{r+1}$. The two questions can therefore select different ranks.

The following stipulated $8\times 8$ matrix makes the tradeoff visible. Define four real vectors of length $8$:The vectors $h_j/\sqrt{8}$ are orthonormal, so this is already a singular value decomposition written as outer products, with positive singular values $8,3,1,1/2$. All entries lie between $7/16$ and $25/16$ and can be displayed as brightnesses. The first component is uniform brightness, the second distinguishes broad blocks, and the last two supply finer alternating patterns. This is a designed arithmetic example, not measured image data.

Here $A_1=h_0{h}_0^T$ has every entry equal to $1$, while $A_2=h_0{h}_0^T+3h_1{h}_1^T/8$ has entries $11/8$ within the two diagonal blocks and $5/8$ in the other two blocks. For instance, $A_{11}=25/16$ becomes $(A_2{)}_{11}=11/8$: its error is $3/16$. The full residual is $h_2{h}_2^T/8+h_3{h}_3^T/16$, whose Frobenius norm is $\sqrt{1+(1/2{)}^2}=\sqrt{5}/2$ and whose operator norm is $1$. Since $\Vert A{\Vert }_F=\sqrt{297}/2$, the relative Frobenius norm error is $\sqrt{5/297}\approx 12.97\%$. Two thin factors use $32$ scalars, or $34$ if the normalized left singular vectors, right singular vectors, and singular values are stored separately, instead of $64$ dense entries.

Retaining three components reduces the Frobenius norm error to $1/2$ while using $48$ scalars in two factors. Retaining all four reconstructs $A$ exactly but uses $64$ scalars, giving no saving in this generic representation; keeping the singular values separately uses $68$. Our particular sign patterns could be encoded more economically, but that would exploit an additional structure beyond the general low-rank approximation. The striking feature is that numerical error and storage can both be read from the decomposition, rather than judged only by how smooth the picture looks.

The constraint is deliberately matrix rank at most $r$. The truncation has $\mathrm{rank}{A}_r=\min (r,\mathrm{rank}A)$. If $\mathrm{rank}A<r$, the best approximation is $A$ itself, with zero error. Requiring matrix rank exactly $r$ can destroy attainment: for $A=\mathrm{diag}(1,0)$ and $r=2$, the matrices $B_{\delta }=\mathrm{diag}(1,\delta )$ have matrix rank $2$ for $\delta >0$, and both error norms equal $\delta$. Their error has infimum zero, but no rank-two matrix attains it. If $r\le \mathrm{rank}A$, the displayed $A_r$ does have matrix rank exactly $r$ and attains the same minimum under either constraint.

Optimality also does not always mean uniqueness. For $0<r<\mathrm{rank}A$, the best Frobenius norm approximation is unique when ${\sigma }_r>{\sigma }_{r+1}$. ^[ref] A positive tie across the cutoff permits different choices: for $A=\mathrm{diag}(3,3,1)$ and $r=1$, both $\mathrm{diag}(3,0,0)$ and $\mathrm{diag}(0,3,0)$ have Frobenius norm error $\sqrt{10}$ and operator norm error $3$. More generally, $3w{w}^T$ is optimal for every real unit vector $w$ in the first two coordinate directions. Rotating which tied direction is kept changes the approximation. Repeated singular values entirely inside the retained group do not change its sum of outer products. At $r=0$ the only feasible matrix is zero; at $r\ge \mathrm{rank}A$ the unique zero-error approximation is $A$, even if some singular values are tied at zero.

A strict gap does not guarantee uniqueness for the operator norm. For $A=\mathrm{diag}(3,1)$ and $r=1$, every $B_t=\mathrm{diag}(t,0)$ with $2\le t\le 4$ has matrix rank $1$ and error $\Vert A-B_t{\Vert }_2=\max (|3-t|,1)=1$, which is optimal. Only $t=3$ minimizes the Frobenius norm, since $\Vert A-B_t{\Vert }_F=\sqrt{(3-t{)}^2+1}$. An unchanged worst direction can hide extra error in another direction under the operator norm.

Finally, these guarantees concern the stated norms and the sole matrix rank constraint. They do not establish that a discarded pattern is noise or irrelevant to a particular task: the finer patterns in our constructed example were put there intentionally. Adding entrywise weights or restrictions on the factors changes the problem, so the Eckart-Young-Mirsky theorem above does not automatically solve it. For historical context, the original 1936 matrix problem is documented in the existing Eckart-Young-Mirsky theorem section.

A real $m\times k$ matrix whose columns are an orthonormal set, equivalently $Q^{T}Q=I_k$. In the complex case use the conjugate transpose instead. This requires $k\le m$; only in the square case does the transpose also provide a two-sided inverse, making $Q$ an orthogonal matrix.

The Moore-Penrose inverse extends the useful part of a matrix inverse to rectangular and singular matrices. A linear system $Ax=b$ may have no exact solution, or may have many. The pseudoinverse resolves both ambiguities: $x_{\ast }=A^{+}b$ is the minimum-norm least-squares solution. First minimize the Euclidean norm $\Vert Ax-b{\Vert }_2$; among all vectors attaining that minimum, choose the one with smallest Euclidean norm $\Vert x{\Vert }_2$. It does not pretend that inconsistent equations can all be satisfied.

For $A\in {\mathbb{C}}^{m\times n}$, its Moore-Penrose inverse is the unique $n\times m$ matrix satisfying the four Penrose equations. The singular value decomposition gives a direct construction:Here $U,V$ are unitary matrices, the star is the conjugate transpose, and ${\Sigma }^{+}$ is the $n\times m$ rectangular diagonal matrix obtained by replacing each nonzero singular value ${\sigma }_j$ by $1/{\sigma }_j$ and leaving zeros at zero. Over the real numbers, use orthogonal matrices and transposes. The pseudoinverse reverses the stretches that can be reversed; it discards output components the original linear map cannot produce and does not invent an input component in its kernel. ^[ref]

The two products have especially clear meanings:They are orthogonal projection matrices: $A{A}^{+}$ selects the producible part of an output, while $A^{+}A$ selects the detectable part of an input. They are generally not identity matrices. For an invertible matrix, both are identities and $A^{+}=A^{-1}$.

This gives a useful order of operations for least-squares solutions: project the requested output onto the column space, then invert the restriction of $A$ from $(\ker A{)}^{\perp }$ to its column space. Adding a kernel vector leaves the residual unchanged but increases the squared Euclidean norm by the Pythagorean theorem in an inner-product space, because $A^{+}b$ is orthogonal to that kernel. The pseudoinverse therefore provides one canonical answer without hiding either lost information or irreconcilable data.

The name records a history with several contributors. E. H. Moore gave an earlier generalization in 1920; M. H. Pearl's 1966 paper records that work, Arne Bjerhammar's 1951 rediscovery for maximal-rank rectangular matrices, and Roger Penrose's independent 1955 treatment. ^[ref] Penrose's original paper proves existence and uniqueness from the four Penrose equations for arbitrary complex matrices, including rectangular ones. ^[ref]

A matrix represents a linear projection when applying it twice has the same effect as applying it once. For an orthogonal projection with the standard Hermitian inner product on ${\mathbb{C}}^m$, the additional condition is that the conjugate transpose equals the original matrix:Thus $P$ is both idempotent and a Hermitian matrix. Over the real numbers this says $P^2=P=P^T$. Its image of a linear map is the retained linear subspace, while its kernel of a linear map is the orthogonal complement: vectors in the image are unchanged, and vectors in the kernel become zero.

For a matrix $A$ with full column rank, projecting onto its column space givesThe invertibility of $A^{†}A$ follows from linear independence of the columns. If those columns are an orthonormal set, $A^{†}A=I$ and $P=A{A}^{†}$. These formulas and the conditions $P^2=P=P^T$ in the real case are developed in MIT 18.06SC, Projections onto Subspaces, pp. 1–3.

For the line spanned by $e=(1,2{)}^T$,In particular, $Pe=e$ and $P(2,-1{)}^T=0$. The eigenvalues are $1$ on the retained direction and $0$ on the removed direction. This exposes the meaning of an orthogonal projection as retaining selected components, rather than stretching them.

Symmetry refers to the specified inner product. In the real weighted setting, the formula becomes $P_M=A(A^{T}MA{)}^{-1}{A}^{T}M$ and satisfies $P_M^2=P_M$ and $P_M^{T}M=M{P}_M$. For the preceding weighted line example,It is not symmetric under the ordinary transpose, but it is an orthogonal projection for the weighted Euclidean inner product. The matrix criterion $P=P^{†}$ should therefore be used with the standard Hermitian inner product, not silently transferred to a different geometry.