= Eckart-Young-Mirsky theorem
{c}
{title2=$\min_{\operatorname{rank}B\leq k}\|A-B\|_F=(\sum_{j>k}\sigma_j^2)^{1/2}$}
For a finite <matrix> with <singular values> in nonincreasing order, truncating its <singular value decomposition> after $k$ terms minimizes the <Frobenius norm> error, whose square is the sum of the discarded squared <singular values>. It also minimizes the <operator norm> error, which equals the next <singular value>, or zero if none remain. See https://math.uchicago.edu/~may/REU2016/REUPapers/Mazeika.pdf[Mazeika, Section 4.3: Low rank approximation].
The problem of replacing a <matrix> by one of lower <matrix rank> is explicit in Eckart and Young's 1936 paper, https://www.cambridge.org/core/journals/psychometrika/article/approximation-of-one-matrix-by-another-of-lower-rank/B29672E1EDD0FA1B7611D4DFAFC321B3[The approximation of one matrix by another of lower rank, Psychometrika 1, 211-218]{ref}. The publication predates the modern <image compression> example, showing how a general matrix question can support later applications.
Back to article page
