Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2006/iii/paper-46/4/a/solution

Write for two rows of the data matrix and . Three possible dissimilarities for continuous variables are the following.
The Euclidean distance is
It is simple and has the usual geometric interpretation when coordinates have comparable units. Its disadvantage is sensitivity to the numerical scales: a change of units or a high-variance variable can dominate it. Strongly correlated coordinates can also give repeated weight to essentially the same information. Squared Euclidean distance is useful in some methods, but is not itself a metric because it need not satisfy the triangle inequality.
The standardized Euclidean distance is
where is the sample variance of coordinate across individuals. It measures differences in standard-deviation units and is invariant under positive coordinate rescalings when the scales are estimated consistently. It is useful for differently measured variables, but requires nonzero sample variances, ignores correlations between coordinates and can give excessive influence to a low-variance noisy variable. With fixed positive scales it is a Euclidean distance after a diagonal change of coordinates, so it is a metric.
The Mahalanobis distance is
where is the pooled sample covariance matrix of all individuals. If is its Cholesky decomposition, then : it is Euclidean distance in whitened coordinates. It adjusts for both scales and correlations, avoiding duplicate weight for highly correlated variables, and is invariant under nonsingular affine transformations when the sample covariance matrix is transformed accordingly. No multivariate normal distribution assumption is required merely to define the distance. However, must be a positive-definite matrix; when there are too few observations or dependent coordinates, its inverse is unavailable. Estimation of can also be unstable and sensitive to outliers. These trade-offs determine whether covariance adjustment is preferable to the simpler distances.

New to topics? Read the docs here!