Past exam of the mathematics course of the University of Cambridge / 2007 / iii / Paper 47 / 2

For an integrable real function $g$, let $R_{-}=\{x:g(x)<0\}$. For every measurable region $R$,The first term is nonnegative because $g\ge 0$ there, and the integral subtracted in the second term is nonpositive. Hence the negative set minimizes the integral. Including or excluding any points where $g=0$ has no effect; changes on null sets do not matter either. This is the minimum-integral decision region principle. Integrability ensures the subtraction is defined, and it holds automatically for the weighted differences of probability density functions used below.

There are two disjoint ways to make an error: an individual from class one falls outside $R_1$, or an individual from class two falls inside $R_1$. The law of total probability givesSince $f_1$ integrates to one, this also equalsThis form isolates the only term changed by choosing the decision region.

Apply the minimum-integral decision region principle to $g(x)={\pi }_2{f}_2(x)-{\pi }_1{f}_1(x)$. The optimal Bayes classifier chooses class one when ${\pi }_1{f}_1(x)>{\pi }_2{f}_2(x)$, and class two for the reverse inequality. Ties can be assigned either way. An everywhere valid discriminant score isWhere both weighted densities are positive one may equivalently use $\log ({\pi }_1{f}_1(x)/({\pi }_2{f}_2(x)))$ and threshold zero. This chooses the largest posterior class probability, because both posterior probabilities have the same positive mixture-density denominator.

With zero cost for correct assignments, the total expected misclassification cost isThe part depending on $R_1$ is the integral of $c(1\mid 2){\pi }_2{f}_2-c(2\mid 1){\pi }_1{f}_1$. Therefore the cost-sensitive Bayes classifier isFor positive costs and weighted densities, the log posterior-odds score from part (ii) now has threshold $\log (c(1\mid 2)/c(2\mid 1))$. Larger cost of sending class one to class two expands the region assigned to class one. The weighted-density comparison also handles zero costs without taking an undefined logarithm.

For common positive-definite covariance, subtracting the two Gaussian log densities cancels the quadratic term. Equal priors leave the linear discriminant analysis scoreHereThusAssign class two if the expression is negative; a tie has zero probability under either continuous Gaussian law.

To compute the error rather than just specify the boundary, put $D^2={\Delta }^T{\Sigma }^{-1}\Delta =27/2$. Within class one the score has mean $D^2/2=27/4$, and within class two mean $-27/4$. In both classes its variance is ${\Delta }^T{\Sigma }^{-1}\Sigma {\Sigma }^{-1}\Delta =D^2$. Hence both conditional error probabilities are $\Phi (-D/2)$, where $\Phi$ is the standard normal distribution function. Equal priors give the equal-covariance Gaussian classification errorFor the specified observation, the score is $\delta (2,1)=27/4>0$, so assign it to class one.