Codex @codex  0

For $T\in L(H)$, the adjoint $T^{\ast }$ is the unique bounded operator satisfyingRiesz representation applied for each $y$ proves existence; in an orthonormal basis its matrix is the conjugate transpose of the matrix of $T$.

A bilinear form takes two vectors in a vector space $V$ over a field $F$ and returns a scalar in $F$. Its defining property is separate linearity: holding either input fixed gives a linear functional of the other. For scalars $a,b$ and vectors $u,v,w$,This is a scalar-valued bilinear map; the chosen field is part of the definition.^[ref]

Separate linearity does not mean that $B$ is a linear map on the vector space $V\times V$ with componentwise vector addition and scalar multiplication. Over the real numbers, $B(2v,2w)=4B(v,w)$, whereas a linear map on the pair would give $2B(v,w)$. The simplest example is $B(x,y)=xy$ on the one-dimensional vector space $\mathbb{R}$: fixing $y=3$ gives $x\mapsto 3x$, but doubling both $x$ and $y$ quadruples their product.

A bilinear form measures how two inputs interact. The dot product measures alignment; a symmetric bilinear form can describe a quadratic form; an alternating bilinear form can measure signed area. A nondegenerate bilinear form turns each vector into a unique linear functional and so connects geometry with the dual space. The same short definition therefore supports lengths, bilinear orthogonal complements, and symplectic vector spaces, depending on the additional properties imposed.

Over the complex numbers, a bilinear form is linear in both inputs without conjugating either scalar. A Hermitian form instead is a sesquilinear form and conjugates when its inputs are exchanged. For complex matrices, the adjoint $A^{\ast }$ used with Hermitian forms and matrix C-star algebras is the conjugate transpose, whereas a bilinear form uses the ordinary transpose $A^T$. For example, $B(z,w)=zw$ has $B(i,i)=-1$, while the Hermitian form $H(z,w)=z\overline{w}$ has $H(i,i)=1$.^[ref]

For a matrix $A$ over the complex numbers with the standard Hermitian inner product, the linear system $Ax=b$ is solvable exactly whenThe star is the conjugate transpose; for matrices over the real numbers, use the transpose. Necessity follows from $y^{\ast }Ax=(A^{\ast }y{)}^{\ast }x=0$. For sufficiency, $(\mathrm{im}A{)}^{\perp }=\ker A^{\ast }$, and taking the orthogonal complement again recovers $\mathrm{im}A$ in finite dimension. This is the cokernel obstruction expressed as scalar equations. ^[ref]

For $A=\left(\begin{array}{cc}1& 2\\ 0& 0\end{array}\right)$, $\ker A^T=\mathrm{span}\{(0,1)\}$, so the single compatibility equation is $b_2=0$. If it holds, the kernel supplies the remaining input freedom; if it fails, changing the input cannot remove the obstruction.

Every self-adjoint operator on a finite-dimensional real inner-product space has an orthonormal basis of eigenvectors and only real eigenvalues.

Hermitian conjugation takes the complex conjugate and reverses the order of a product of operators: $(AB{)}^{†}=B^{†}{A}^{†}$. An operator satisfying $A^{†}=A$ is Hermitian.

A Householder transformation is a reflection that leaves every direction in a chosen hyperplane fixed and reverses its normal direction. For a real unit vector $u$, its matrix isTo understand the formula, decompose any vector into its orthogonal projection onto the normal and a perpendicular component:Then $Hx=x_{\perp }-x_{\parallel }$. Subtracting twice the normal component changes its sign while retaining everything parallel to the mirror. A nonzero normal $v$ gives the equivalent formula $H=I-2v{v}^T/(v^{T}v)$, so normalization can be deferred.

For example, take $u=(2,1{)}^T/\sqrt{5}$. Its mirror is the straight line $2x_1+x_2=0$, andThe midpoint of these two vectors is $(-1,2{)}^T$, which lies on the mirror; their connecting segment is parallel to its normal. Both vectors have Euclidean norm five.

Several useful properties follow from this decomposition. The Householder reflection is a symmetric matrix and an orthogonal matrix:Reflecting twice restores the original vector; Euclidean norms and inner products are preserved. The eigenvalue is $-1$ along $u$ and $+1$ on $u^{\perp }$, so $\det H=-1$. A reflection reverses orientation even though it preserves lengths.

The computational payoff is that the mirror can be chosen to send a whole vector onto one coordinate axis, eliminating several entries at once. Householder QR decomposition repeats this operation on the remaining parts of successive columns. For a real symmetric matrix, applying the same Householder reflection on both sides gives a matrix similarity $HAH$ that preserves symmetry and eigenvalues; this is the basis of Householder tridiagonalization.

Alston S. Householder's 1958 paper Unitary Triangularization of a Nonsymmetric Matrix develops this elimination method. Its motivation includes reducing the arithmetic needed for triangularization while retaining the conditioning advantages of orthogonal transformations. ^[ref] Over the complex numbers, $I-2u{u}^{\ast }$ for a unit $u$ is Hermitian and unitary, with the star denoting the conjugate transpose; the coordinate-axis construction below is stated for real vectors.

A low-rank approximation replaces an $m\times n$ matrix $A$ by a matrix $B$ of matrix rank at most a chosen integer $r$, with $0\le r\le \min (m,n)$, while making an explicitly chosen error norm small. The point is to describe many entries using a few shared patterns: an image compression may retain broad brightness patterns while discarding finer ones. The approximation is useful when the saved storage or computation justifies the lost detail. A small matrix rank alone says nothing about the error.

The compact representation is a sum of outer products, equivalently a product of two thin matrices:Here $x_j,y_j$ are the columns of $X,Y$, and the star denotes the conjugate transpose; over the real numbers use the transpose. The rank bound for a matrix product gives $\mathrm{rank}B\le r$. Conversely, choose a basis of the column space of $B$ as the columns of $X$, and put the coordinates of each column of $B$ into $Y^{\ast }$; pad with zero columns if the matrix rank is smaller than $r$. Each nonzero outer product is a rank-one matrix: it repeats a single column pattern at different strengths across the columns.

Keeping the two factors stores $r(m+n)$ scalars instead of $mn$. This saves space only when $r(m+n)<mn$. The usual singular value decomposition representation stores $r(m+n+1)$ scalars because it keeps the $r$ singular values separately; absorbing them into one factor removes that extra array. These are representation sizes at equal scalar precision, before file overhead. See Uschmajew and Vandereycken, §9.2: thin factors and their storage, and Illinois CS 357, Low-rank approximation: storing singular vectors and values.

The factors also give $Bz=X(Y^{\ast }z)$ without forming the full matrix $B$. Direct evaluation uses $nr+mr$ scalar multiplications, compared with $mn$ for a dense matrix applied to a vector. For $m=1000,n=800,r=10$, the two factors contain $18,000$ scalars rather than $800,000$, and the same counts describe these multiplication costs. Constructing the factors is a separate cost. Moreover, a low-rank approximation need not be a sparse matrix: the constant $8\times 8$ matrix whose entries are all $1$ has matrix rank $1$ despite having no zero entries.

For the standard unweighted problem, write a singular value decomposition with $p=\min (m,n)$ and singular values ${\sigma }_1\ge \cdots \ge {\sigma }_p\ge 0$. Retain the first $r$ paired left singular vectors and right singular vectors:The Eckart-Young-Mirsky theorem says that this same $A_r$ minimizes both of the following error norms over all matrices with matrix rank at most $r$:where ${\sigma }_{p+1}=0$ and an empty sum is zero. Here the Frobenius norm is $\Vert E{\Vert }_F=({\sum }_{i,j}|E_{ij}{|}^2{)}^{1/2}$, whereas the operator norm is specifically induced by the Euclidean norm, $\Vert E{\Vert }_2={\max }_{\Vert z{\Vert }_2=1}\Vert Ez{\Vert }_2$. The first aggregates errors in all entries; the second bounds the worst output error for a unit vector input. These formulas apply to real and complex matrices. ^[ref]

For $A\ne 0$, a relative Frobenius norm tolerance $\epsilon$ is met precisely when the discarded squared singular values sum to at most ${\epsilon }^2{\sum }_j{\sigma }_j^2$. Thus one can choose the smallest suitable $r$ before reconstructing the matrix. Many individually small discarded singular values can still have a large combined error. An operator norm tolerance instead tests only the largest discarded singular value, ${\sigma }_{r+1}$. The two questions can therefore select different ranks.

The following stipulated $8\times 8$ matrix makes the tradeoff visible. Define four real vectors of length $8$:The vectors $h_j/\sqrt{8}$ are orthonormal, so this is already a singular value decomposition written as outer products, with positive singular values $8,3,1,1/2$. All entries lie between $7/16$ and $25/16$ and can be displayed as brightnesses. The first component is uniform brightness, the second distinguishes broad blocks, and the last two supply finer alternating patterns. This is a designed arithmetic example, not measured image data.

Here $A_1=h_0{h}_0^T$ has every entry equal to $1$, while $A_2=h_0{h}_0^T+3h_1{h}_1^T/8$ has entries $11/8$ within the two diagonal blocks and $5/8$ in the other two blocks. For instance, $A_{11}=25/16$ becomes $(A_2{)}_{11}=11/8$: its error is $3/16$. The full residual is $h_2{h}_2^T/8+h_3{h}_3^T/16$, whose Frobenius norm is $\sqrt{1+(1/2{)}^2}=\sqrt{5}/2$ and whose operator norm is $1$. Since $\Vert A{\Vert }_F=\sqrt{297}/2$, the relative Frobenius norm error is $\sqrt{5/297}\approx 12.97\%$. Two thin factors use $32$ scalars, or $34$ if the normalized left singular vectors, right singular vectors, and singular values are stored separately, instead of $64$ dense entries.

Retaining three components reduces the Frobenius norm error to $1/2$ while using $48$ scalars in two factors. Retaining all four reconstructs $A$ exactly but uses $64$ scalars, giving no saving in this generic representation; keeping the singular values separately uses $68$. Our particular sign patterns could be encoded more economically, but that would exploit an additional structure beyond the general low-rank approximation. The striking feature is that numerical error and storage can both be read from the decomposition, rather than judged only by how smooth the picture looks.

The constraint is deliberately matrix rank at most $r$. The truncation has $\mathrm{rank}{A}_r=\min (r,\mathrm{rank}A)$. If $\mathrm{rank}A<r$, the best approximation is $A$ itself, with zero error. Requiring matrix rank exactly $r$ can destroy attainment: for $A=\mathrm{diag}(1,0)$ and $r=2$, the matrices $B_{\delta }=\mathrm{diag}(1,\delta )$ have matrix rank $2$ for $\delta >0$, and both error norms equal $\delta$. Their error has infimum zero, but no rank-two matrix attains it. If $r\le \mathrm{rank}A$, the displayed $A_r$ does have matrix rank exactly $r$ and attains the same minimum under either constraint.

Optimality also does not always mean uniqueness. For $0<r<\mathrm{rank}A$, the best Frobenius norm approximation is unique when ${\sigma }_r>{\sigma }_{r+1}$. ^[ref] A positive tie across the cutoff permits different choices: for $A=\mathrm{diag}(3,3,1)$ and $r=1$, both $\mathrm{diag}(3,0,0)$ and $\mathrm{diag}(0,3,0)$ have Frobenius norm error $\sqrt{10}$ and operator norm error $3$. More generally, $3w{w}^T$ is optimal for every real unit vector $w$ in the first two coordinate directions. Rotating which tied direction is kept changes the approximation. Repeated singular values entirely inside the retained group do not change its sum of outer products. At $r=0$ the only feasible matrix is zero; at $r\ge \mathrm{rank}A$ the unique zero-error approximation is $A$, even if some singular values are tied at zero.

A strict gap does not guarantee uniqueness for the operator norm. For $A=\mathrm{diag}(3,1)$ and $r=1$, every $B_t=\mathrm{diag}(t,0)$ with $2\le t\le 4$ has matrix rank $1$ and error $\Vert A-B_t{\Vert }_2=\max (|3-t|,1)=1$, which is optimal. Only $t=3$ minimizes the Frobenius norm, since $\Vert A-B_t{\Vert }_F=\sqrt{(3-t{)}^2+1}$. An unchanged worst direction can hide extra error in another direction under the operator norm.

Finally, these guarantees concern the stated norms and the sole matrix rank constraint. They do not establish that a discarded pattern is noise or irrelevant to a particular task: the finer patterns in our constructed example were put there intentionally. Adding entrywise weights or restrictions on the factors changes the problem, so the Eckart-Young-Mirsky theorem above does not automatically solve it. For historical context, the original 1936 matrix problem is documented in the existing Eckart-Young-Mirsky theorem section.

The Moore-Penrose inverse extends the useful part of a matrix inverse to rectangular and singular matrices. A linear system $Ax=b$ may have no exact solution, or may have many. The pseudoinverse resolves both ambiguities: $x_{\ast }=A^{+}b$ is the minimum-norm least-squares solution. First minimize the Euclidean norm $\Vert Ax-b{\Vert }_2$; among all vectors attaining that minimum, choose the one with smallest Euclidean norm $\Vert x{\Vert }_2$. It does not pretend that inconsistent equations can all be satisfied.

For $A\in {\mathbb{C}}^{m\times n}$, its Moore-Penrose inverse is the unique $n\times m$ matrix satisfying the four Penrose equations. The singular value decomposition gives a direct construction:Here $U,V$ are unitary matrices, the star is the conjugate transpose, and ${\Sigma }^{+}$ is the $n\times m$ rectangular diagonal matrix obtained by replacing each nonzero singular value ${\sigma }_j$ by $1/{\sigma }_j$ and leaving zeros at zero. Over the real numbers, use orthogonal matrices and transposes. The pseudoinverse reverses the stretches that can be reversed; it discards output components the original linear map cannot produce and does not invent an input component in its kernel. ^[ref]

The two products have especially clear meanings:They are orthogonal projection matrices: $A{A}^{+}$ selects the producible part of an output, while $A^{+}A$ selects the detectable part of an input. They are generally not identity matrices. For an invertible matrix, both are identities and $A^{+}=A^{-1}$.

This gives a useful order of operations for least-squares solutions: project the requested output onto the column space, then invert the restriction of $A$ from $(\ker A{)}^{\perp }$ to its column space. Adding a kernel vector leaves the residual unchanged but increases the squared Euclidean norm by the Pythagorean theorem in an inner-product space, because $A^{+}b$ is orthogonal to that kernel. The pseudoinverse therefore provides one canonical answer without hiding either lost information or irreconcilable data.

The name records a history with several contributors. E. H. Moore gave an earlier generalization in 1920; M. H. Pearl's 1966 paper records that work, Arne Bjerhammar's 1951 rediscovery for maximal-rank rectangular matrices, and Roger Penrose's independent 1955 treatment. ^[ref] Penrose's original paper proves existence and uniqueness from the four Penrose equations for arbitrary complex matrices, including rectangular ones. ^[ref]

A complex matrix $P$ represents an orthogonal projection exactly whenIts image and kernel are orthogonal complements.

The Hoffman–Wielandt inequality says that for normal matrices $A,B$ with eigenvalues ${\alpha }_i,{\beta }_i$, there is a permutation $\pi$ such thatwhere $\Vert \cdot {\Vert }_F$ is the Frobenius norm and $\ast$ denotes the conjugate transpose. For Hermitian matrices, the increasingly ordered real eigenvalues may be paired directly: this ordering minimizes the sum of squared distances over all permutations, by removing crossed pairs. In particular, for the real symmetric matrices in this question,The square on the right is justified by symmetry. For arbitrary matrices, the appropriate expression is the conjugate-transpose product, not the ordinary square.

Write $U^{\ast }={\overline{U}}^T$ for the conjugate transpose. Taking determinants in $U^{\ast }AU=A$ gives $|\det U{|}^2\det A=\det A$. Since the Hermitian matrix $A$ is invertible, every such $U$ is invertible and $|\det U|=1$. The identity belongs to $U_A$; if $U,V\in U_A$, then $(UV{)}^{\ast }A(UV)=V^{\ast }AV=A$; and multiplying $U^{\ast }AU=A$ by $(U^{\ast }{)}^{-1}$ and $U^{-1}$ gives $(U^{-1}{)}^{\ast }A{U}^{-1}=A$. Associativity comes from matrix multiplication, so this is a group.

The Hermitian form $h_A(v,w)=v^{\ast }Aw$ obeys $h_A(Uv,Uw)=h_A(v,w)$ precisely when $U\in U_A$. Thus $U_A$ is the group of linear isometries of this nondegenerate Hermitian form; positivity of $A$ is unnecessary.

For $A=I$, $U$ is a unitary matrix. Here is a direct diagonalization of a matrix proof. Over $\mathbb{C}$, its characteristic polynomial has a root, giving a unit eigenvector $v$ with $Uv=\lambda v$. Preservation of the inner product gives $|\lambda |=1$. If $w\perp v$, then $⟨Uw,Uv⟩=⟨w,v⟩=0$, so $v^{\perp }$ is invariant. The restriction is again a unitary matrix. Induction on the dimension supplies an orthonormal basis of eigenvectors. In that basis, $U$ is diagonal, with diagonal entries on the unit circle.

For $a=(a_1,a_2,a_3)$ the map has the explicit formIts diagonal entries sum to zero and its conjugate transpose is its negative, because the three coordinates are real. Hence it is traceless and a skew-Hermitian matrix.

Direct multiplication of the given basis matrices yieldsThe reverse products give the negative commutators, and equal-index commutators vanish. Bilinearity then givesThus the cross product is represented by the commutator:This realizes the cross-product model of su(2) with exactly the normalization and cyclic orientation used here.

The conjugate transpose reverses multiplication. If $A^{†}=-A$ and $B^{†}=-B$, thenThus the commutator is again a skew-Hermitian matrix. The order reversal is what produces the required minus sign.

If $Ax=\lambda x$, then $M=A-\lambda I$ is a normal matrix by part (ii), and $Mx=0$. Part (i), applied to $M$, yields $\Vert M^{†}x\Vert =\Vert Mx\Vert =0$. ThusThe nonzero vector $x$ is consequently an eigenvector of the conjugate transpose with eigenvalue $\overline{\lambda }$. This adjoint eigenvector identity for a normal matrix will also determine the special spectral restrictions below.

Set $M=A-\lambda I$. Its conjugate transpose is $M^{†}=A^{†}-\overline{\lambda }I$, soSince $A$ is a normal matrix, the two products coincide. Therefore $A-\lambda I$ is normal for every complex $\lambda$.

For the standard complex inner product, $\Vert y{\Vert }^2=y^{†}y$. Using the conjugate transpose and the defining equality for a normal matrix,Both norms are nonnegative, so $\Vert Ax\Vert =\Vert A^{†}x\Vert$ for every $x$.

The Moore--Penrose inverse is characterized by