 Eckart-Young-Mirsky theorem

For a finite matrix with singular values in nonincreasing order, truncating its singular value decomposition after $k$ terms minimizes the Frobenius norm error, whose square is the sum of the discarded squared singular values. It also minimizes the operator norm error, which equals the next singular value, or zero if none remain. See Mazeika, Section 4.3: Low rank approximation.

The problem of replacing a matrix by one of lower matrix rank is explicit in Eckart and Young's 1936 paper, ^[ref]. The publication predates the modern image compression example, showing how a general matrix question can support later applications.