= Clustering
{disambiguate=machine learning}
<Clustering (machine learning)> groups observations according to a specified notion of similarity or shared structure. Unlike <supervised learning>, it need not begin with known class labels. The representation, distance and objective determine what counts as a group: nearby numerical values, a connected dense region and membership in a probabilistic component are different criteria.
In <K-means clustering>, $K$ centers $m_j$ minimize the within-group squared-distance objective
$$
J=\sum_{j=1}^K\sum_{i\in C_j}\lVert x_i-m_j\rVert^2.
$$
For fixed groups the minimizing center is their mean; for fixed centers each point is assigned to a nearest center. These elementary minimizations explain the alternating updates. <Stuart P. Lloyd>'s original quantization paper develops the corresponding nearest-representative and mean conditions for scalar signal reconstruction.<Least Squares Quantization in PCM by Stuart P. Lloyd>[Original paper, necessary conditions for least-squares quantizers and iterative construction]{ref}
For an independent one-dimensional example, the data $0,2,8,10$ can be grouped as $\{0,2\}$ and $\{8,10\}$, with means $1,9$ and objective $4$. A single center at $5$ gives objective $68$. Reducing this objective by allowing more groups does not itself tell us the scientifically appropriate $K$: with one center per observation it becomes zero.
<Clustering (machine learning)> describes the chosen representation, not an automatic discovery of natural kinds. Changing units can change distances; elongated or nested groups can challenge a squared-distance criterion; different initial centers can produce different fitted solutions. Stability, the scientific question and knowledge of how observations were collected help determine whether a partition is informative.
Back to article page
