= Singular value
{title2=$\sigma_j\geq0$}
{wiki}
The <singular values> of a <matrix> measure stretching of <Euclidean norm>. They are the nonnegative <square roots> of the <eigenvalues> of $A^*A$, counting the $\min(m,n)$ diagonal slots of the <singular value decomposition>; when $n>m$, $A^*A$ has additional zero <eigenvalues>. Nonnegativity follows from $x^*A^*Ax=\|Ax\|_2^2\geq0$, so $A^*A$ is a <positive semidefinite matrix>. For the preceding $2\times2$ <matrix>,
$$
A^TA=\frac1{25}\begin{pmatrix}241&288\\288&409\end{pmatrix},\qquad
AA^T=\begin{pmatrix}25&0\\0&1\end{pmatrix}.
$$
Both have <eigenvalues> $25,1$, giving <singular values> $5,1$. In particular,
$$
\sigma_1=\max_{\|x\|_2=1}\|Ax\|_2=\|A\|_2.
$$
Here the <operator norm> uses <Euclidean norm>: no <unit vector> stretches more than fivefold. Zero <singular values> identify lost directions; the number of positive <singular values> equals the <matrix rank>. This summarizes both amplification and information loss with nonnegative numbers.https://math.mit.edu/~gs/linearalgebra/ila5/SVD_Slides.pdf[Verma, slides 7–8: singular values and maximum stretching]{ref}
Small positive <singular values> also explain sensitivity in an <inverse problem>. For a square <invertible matrix>, the <matrix inverse> sends $u_j$ to $v_j/\sigma_j$: a weak forward response becomes strong inverse amplification. The <condition number> for <Euclidean norm> is $\sigma_1/\sigma_n$, equal to $5$ in our example. To see a larger effect, keep the same <right singular vectors> but change the second <singular value> to $\varepsilon=0.01$:
$$
A_\varepsilon=\begin{pmatrix}3&4\\-4\varepsilon/5&3\varepsilon/5\end{pmatrix},\qquad
\delta b=0.001u_2,\qquad
\delta x=A_\varepsilon^{-1}\delta b=0.1v_2=(-0.08,0.06)^T.
$$
The output perturbation has <Euclidean norm> $0.001$, but the recovered input changes by $0.1$. Its hundredfold amplification follows from the direction of the perturbation, not just its size. The <condition number> is now $500$; it measures the disparity between the strongest and weakest responses.https://www.cs.cornell.edu/courses/cs322/2007sp/notes/condition.pdf[Marschner, Condition numbers, §2: inverse amplification and singular values]{ref}
Back to article page
