Singular value (${\sigma }_j\ge 0$)

The singular values of a matrix measure stretching of Euclidean norm. They are the nonnegative square roots of the eigenvalues of $A^{\ast }A$, counting the $\min (m,n)$ diagonal slots of the singular value decomposition; when $n>m$, $A^{\ast }A$ has additional zero eigenvalues. Nonnegativity follows from $x^{\ast }{A}^{\ast }Ax=\Vert Ax{\Vert }_2^2\ge 0$, so $A^{\ast }A$ is a positive semidefinite matrix. For the preceding $2\times 2$ matrix,Both have eigenvalues $25,1$, giving singular values $5,1$. In particular,Here the operator norm uses Euclidean norm: no unit vector stretches more than fivefold. Zero singular values identify lost directions; the number of positive singular values equals the matrix rank. This summarizes both amplification and information loss with nonnegative numbers.^[ref]

Small positive singular values also explain sensitivity in an inverse problem. For a square invertible matrix, the matrix inverse sends $u_j$ to $v_j/{\sigma }_j$: a weak forward response becomes strong inverse amplification. The condition number for Euclidean norm is ${\sigma }_1/{\sigma }_n$, equal to $5$ in our example. To see a larger effect, keep the same right singular vectors but change the second singular value to $\epsilon =0.01$:The output perturbation has Euclidean norm $0.001$, but the recovered input changes by $0.1$. Its hundredfold amplification follows from the direction of the perturbation, not just its size. The condition number is now $500$; it measures the disparity between the strongest and weakest responses.^[ref]

For a real matrix $A$, let $B=A^{\mathsf{T}}A$. It is positive semidefinite, and the spectral theorem for real symmetric matrices gives $x^{\mathsf{T}}Bx\ge {\lambda }_{\min }\Vert x{\Vert }^2$. If ${\lambda }_{\min }>0$, rescaling any longer vector to unit length decreases its image norm, so minimizers lie on the unit sphere. Lagrange multipliers applied to $x^{\mathsf{T}}Bx$ with $\Vert x\Vert =1$ give $Bx=\lambda x$. Thus the minimizers are exactly the unit vectors in the least eigenspace.

If ${\lambda }_{\min }=0$, the minimum is zero and the full argmin consists of all $x\in \ker A$ with $\Vert x\Vert \ge 1$. Indeed, $\ker (A^{\mathsf{T}}A)=\ker A$ because $x^{\mathsf{T}}{A}^{\mathsf{T}}Ax=\Vert Ax{\Vert }^2$. Restricting the minimizers to the sphere would omit longer kernel vectors in this case. For a ball constraint $\Vert x\Vert \le 1$, the argmin is always $\ker A$ intersected with the ball.

For equally sized matrices, corresponding ordered singular values differ by at most the spectral norm of the perturbation. Indeed, let $B_{j-1}$ be a best approximation of matrix rank at most $j-1$. The Eckart-Young-Mirsky theorem and the triangle inequality give ${\sigma }_j(A)\le \Vert A-B_{j-1}{\Vert }_2\le \Vert A-B{\Vert }_2+{\sigma }_j(B)$. Interchanging $A,B$ proves the bound.

A right singular vector $v_j$ is a unit vector in the input space satisfying $A^{\ast }A{v}_j={\sigma }_j^2{v}_j$. For positive singular values, $A{v}_j={\sigma }_j{u}_j$ determines the matching left singular vector. In the ellipse example, $A{v}_1=(5,0{)}^T$ is not parallel to $v_1$: a right singular vector need not be an eigenvector of $A$. Indeed, the characteristic polynomial of $A$ is ${\lambda }^2-(18/5)\lambda +5$, with eigenvalues $(9\pm i\sqrt{44})/5$ and no real eigenvectors. Nevertheless, its real right singular vectors give a complete orthonormal basis. Eigenvectors describe preserved directions; right singular vectors describe orthogonal input channels of stretching.^[ref]

An arbitrary input mixes these channels. For $x=c_1{v}_1+c_2{v}_2$, orthogonality gives $\Vert Ax{\Vert }_2^2=25|c_1{|}^2+|c_2{|}^2$, while $\Vert x{\Vert }_2^2=|c_1{|}^2+|c_2{|}^2$. Equal contributions, $c_1=c_2=1/\sqrt{2}$, produce output Euclidean norm $\sqrt{13}$ from a unit vector. Concentrating the entire input along $v_1$ produces the maximum $5$. This is why the right singular vectors give a direct reading of how any input will behave. For a compact operator, the same definition describes the positive terms of its singular value system.

A left singular vector $u_j$ is a unit vector in the output space satisfying $A{A}^{\ast }{u}_j={\sigma }_j^2{u}_j$. For positive singular values, the pairing also satisfies $A^{\ast }{u}_j={\sigma }_j{v}_j$. Thus $u_1=(1,0{)}^T$ and $u_2=(0,1{)}^T$ describe the output axes of the preceding ellipse, while the right singular vectors describe its input directions. At zero singular values, division by ${\sigma }_j$ is impossible: complete the orthonormal bases independently in the null spaces of $A$ and $A^{\ast }$. Repeated singular values allow different matched orthonormal bases within the corresponding eigenspaces. The same positive pairing extends to a compact operator through its singular value system.

Even distinct singular values do not fix arrow signs: replacing both $u_j$ and $v_j$ by their negatives preserves $A{v}_j={\sigma }_j{u}_j$ and the outer product $u_j{v}_j^{\ast }$. Reversing only one arrow would change that term's sign. The paired directions, rather than independently chosen arrow orientations, encode the matrix.