Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 46 / 2

The inequality in the printed rule is reversed. With the stated definition of $L$, a larger score favors group one. For example, in one dimension take ${\mu }_1=1$, ${\mu }_2=-1$, $\Sigma =1$, equal priors and $x=1$. The score is $2$, and $f_1(1)>f_2(1)$, whereas the printed rule assigns group two. The correct Bayes classifier follows from minimizing the error cost as follows.

Let $R$ be the region assigned to group one, let $f_i$ be its multivariate normal density, and let each misclassification cost $c>0$, with zero cost for correct classification. The expected cost isThe permitted integral-minimization result therefore selects $R=\{x:{\pi }_1{f}_1(x)>{\pi }_2{f}_2(x)\}$. Points of equality may be assigned to either group without changing the risk. Equivalently, the Bayes classifier chooses the larger posterior probability.

For positive priors and a common positive-definite covariance matrix, the normalization constants cancel in the likelihood ratio. Expanding the two quadratic forms givesThus the correct Gaussian Bayes classifier, choosing group one on ties, isThis proves the intended classification result with the necessary correction. Equal error costs remove $c$ from the decision, while prior odds determine the threshold.

Use a multivariate normal distribution within each species, a common positive-definite covariance matrix, equal prior odds and equal misclassification costs. Assume the training observations are independent and representative of the populations. Substituting the sample means and pooled sample covariance matrix gives a fitted linear discriminant analysis rule, rather than a rule with known population parameters.

The determinant of the pooled covariance is $8$, soThe midpoint of the sample means is $(4,6{)}^T$, whose scalar product with $L$ is $-11/2$. Hence the fitted log likelihood ratio isAssign fattus when $q(x)>0$ and apathus when $q(x)<0$; either assignment is optimal on $q(x)=0$ under equal priors. The separating line is $x_2=0.4x_1+4.4$, with fattus below it. As a sign check, the scores at the two sample means are $19/4$ and $-19/4$, respectively. This uses the corrected Gaussian Bayes classifier from (a).

Evaluating the fitted linear discriminant analysis score givesThe fifth individual lies exactly on the discriminant boundary. Its two fitted posterior probabilities are equal, so the equal-prior Bayes classifier cannot prefer either species. A specified tie convention could assign it to one species, but such an assignment would not be evidence favoring that species.

In the sketch the solid line is the equal-prior boundary. The dashed line is the prior-dependent Gaussian discriminant boundary for the altered priors in (iii).

Now the prior odds are ${\pi }_f/{\pi }_a=2$. The fitted log posterior odds are $q(x)+\log 2$, so the Gaussian Bayes classifier assigns fattus when $q(x)>-\log 2$. The prior-dependent Gaussian discriminant boundary becomesIt moves upwards parallel to the original line by $0.8\log 2\simeq 0.5545$, expanding the fattus region. This is the dashed line in the sketch. The fifth individual's fitted log likelihood ratio remains zero, but its posterior odds become $2:1$, giving posterior fattus probability $2/3$.

The fifth individual is now classified as fattus. For completeness, the other four classifications remain apathus, fattus, apathus and fattus: their scores lie on the same side of the new threshold as before.