Past exam of the numerical analysis course of the University of Oxford / 2021 / 2 / b

Retain the first $r$ terms in the singular value decomposition:The Eckart-Young-Mirsky theorem says that this truncated singular value decomposition is a best low-rank approximation in the spectral norm. The remaining singular values of the difference are ${\sigma }_{r+1},\dots ,{\sigma }_n$, together with zeros, soFor $r\ge 1$ the displayed matrix has matrix rank exactly $r$, since every retained singular value is positive. For $r=0$, the only matrix of matrix rank zero is the zero matrix, and the same formula gives error ${\sigma }_1$.

Take the identity matrixBoth $B_1$ and $B_2$ have matrix rank one, andThey are distinct and both attain the minimum spectral norm error from the preceding part. Hence a best low-rank approximation of rank one need not be unique. More generally, every orthogonal projection $u{u}^T$ with $\Vert u{\Vert }_2=1$ gives another such minimizer.

No. Distinct singular values do not force uniqueness in the spectral norm. For an explicit example takeThe singular values of $A$ are $2>1>0$. Every $B_t$ has matrix rank one, and its error in the spectral norm isThus there are infinitely many best rank-one approximations even with a strict gap between the singular values.

The same mechanism works for every $1\le r<n$ under the question's positive singular values. Starting with $A_r$, defineWhen the singular values are distinct, ${\sigma }_1+t\ge {\sigma }_1-{\sigma }_{r+1}>0$, so every $B_t$ has matrix rank $r$, whileThe spectral norm measures only the largest remaining singular value, leaving room to change retained components without increasing that maximum. This is the nonuniqueness of spectral-norm low-rank approximation. The exceptional case $r=0$ is unique for the elementary reason given above.