Bayes classifier

Under zero-one loss, a Bayes classifier assigns an observation to a class of greatest posterior probability. For two classes with prior probabilities ${\pi }_0,{\pi }_1$ and densities $f_0,f_1$, it chooses class one exactly when ${\pi }_1{f}_1(x)\ge {\pi }_0{f}_0(x)$.

With zero cost for correct classifications, assign class one when the displayed inequality holds and class two otherwise. It compares conditional expected losses; ties may be resolved arbitrarily. For positive costs and densities, the log likelihood ratio threshold is $\log (c(1\mid 2){\pi }_2/(c(2\mid 1){\pi }_1))$.

For integrable real $g$, including points where $g<0$ decreases the integral and including points where $g>0$ increases it. Thus the negative set minimizes the integral, up to null sets and arbitrary choices where $g=0$. This elementary signed-integral fact derives the Bayes classifier and cost-sensitive Bayes classifier.

For two multivariate normal densities, the Bayes log posterior odds are a quadratic polynomial. Equal covariance matrices cancel the quadratic term and give linear discriminant analysis; unequal covariance matrices give quadratic discriminant analysis.

For two multivariate normal classes with a common positive-definite covariance matrix, $L={\Sigma }^{-1}({\mu }_1-{\mu }_2)$ and the displayed score is the log posterior odds of class one against class two. With equal error costs, positive scores favor class one. Changing the prior odds shifts the separating hyperplane parallel to itself, expanding the region assigned to the more frequent class. On the boundary either decision has the same conditional risk.

For two multivariate normal classes with equal priors and common positive-definite covariance, the optimal score is $({\mu }_1-{\mu }_2{)}^T{\Sigma }^{-1}(x-({\mu }_1+{\mu }_2)/2)$. Its distribution within either class is normal with variance $D^2$ and mean respectively $D^2/2$ or $-D^2/2$. Both class error probabilities are therefore the displayed normal tail.

A binary Bayes classifier under zero-one loss is unique up to null sets when the posterior class probabilities tie only on a null set. For two distinct nonsingular Gaussian distributions, the tie set is the zero set of a nonzero quadratic polynomial and has Lebesgue measure zero.