Past exam of the mathematics course of the University of Cambridge / 2007 / iii / Paper 47 / 4 / a

For rows $x_i,x_j$, three useful choices are Euclidean distance, standardized Euclidean distance, and Mahalanobis distance:The Euclidean choice is simple, rotation invariant and appropriate when the original coordinate scales carry comparable meaning. It is sensitive to scale changes, may be dominated by a high-variance variable and gives correlated or duplicate variables repeated influence. Squared Euclidean dissimilarity is sometimes used algorithmically, but removing the square root generally loses the triangle inequality.

The standardized Euclidean choice removes marginal units using fixed positive sample scales $s_r$. It gives equal marginal weight to variables, but ignores correlations and can magnify noise when an estimated variance is small. Constant variables require omission or an explicitly chosen positive external scale.

The Mahalanobis choice incorporates the full positive-definite sample covariance $S$, so it is Euclidean distance after whitening. It accounts for both scale and redundancy and is invariant under an invertible change of coordinates when $S$ is transformed consistently. However estimating $S$ requires enough observations and can be sensitive to extreme values; inversion is unstable near singularity and unavailable when $S$ is singular. With fixed positive scales or positive-definite $S$, all three square-root expressions are genuine metrics on the measurement space. Their different geometries encode different notions of which individuals should be considered similar.