= Small singular values in the Moore-Penrose inverse
{title2=$\|A^+\|_2=1/\sigma_{\min,+}$}
For a nonzero <matrix> $A$, the <operator norm> induced by the <Euclidean norm> satisfies $\|A^+\|_2=1/\sigma_{\min,+}$, where $\sigma_{\min,+}$ is its smallest positive <singular value>. The <singular value decomposition> makes the danger visible: a small nonzero stretch becomes a large amplification of errors when inverted.
For instance, when $\varepsilon>0$,
$$
A_\varepsilon=\begin{pmatrix}1&0\\0&\varepsilon\end{pmatrix},
\qquad
A_\varepsilon^+=\begin{pmatrix}1&0\\0&1/\varepsilon\end{pmatrix}.
$$
A data perturbation $(0,\eta)^T$ changes the reconstructed input by $(0,\eta/\varepsilon)^T$. At $\varepsilon=0$, however, the <pseudoinverse> is $\operatorname{diag}(1,0)$: exactly zero and merely small are different cases. Thus the <pseudoinverse> need not be <continuous> across a change of <matrix rank>. With uncertain data, <truncated singular value decomposition> or <Tikhonov regularization> can trade exact inversion of weak directions for stability; the cutoff or penalty adds a modelling choice.
Back to article page
