Clustering (machine learning)

Clustering groups observations according to a specified notion of similarity or shared structure. Unlike supervised learning, it need not begin with known class labels. The representation, distance and objective determine what counts as a group: nearby numerical values, a connected dense region and membership in a probabilistic component are different criteria.

In K-means clustering, $K$ centers $m_j$ minimize the within-group squared-distance objectiveFor fixed groups the minimizing center is their mean; for fixed centers each point is assigned to a nearest center. These elementary minimizations explain the alternating updates. Stuart P. Lloyd's original quantization paper develops the corresponding nearest-representative and mean conditions for scalar signal reconstruction.^[ref]

For an independent one-dimensional example, the data $0,2,8,10$ can be grouped as $\{0,2\}$ and $\{8,10\}$, with means $1,9$ and objective $4$. A single center at $5$ gives objective $68$. Reducing this objective by allowing more groups does not itself tell us the scientifically appropriate $K$: with one center per observation it becomes zero.

Clustering describes the chosen representation, not an automatic discovery of natural kinds. Changing units can change distances; elongated or nested groups can challenge a squared-distance criterion; different initial centers can produce different fitted solutions. Stability, the scientific question and knowledge of how observations were collected help determine whether a partition is informative.