Linear discriminant analysis

LDA assumes class-conditional Gaussian distributions with a shared covariance matrix, producing affine log odds.

For positive-definite within-group scatter $W$ and between-group scatter $B$, directions maximizing $a^{T}Ba/(a^{T}Wa)$ solve the displayed generalized eigenvalue problem. Put $u=W^{1/2}a$; the quotient becomes the Rayleigh quotient of $W^{-1/2}B{W}^{-1/2}$. Its ordered orthonormal eigenvectors give directions orthogonal in the $W$ inner product. There are at most $g-1$ nonzero directions, where $g$ is the number of groups. These directions summarize separation for linear discriminant analysis.

Kernel linear discriminant analysis performs regularized linear discriminant analysis in a Reproducing kernel Hilbert space, using kernel evaluations to obtain nonlinear decision boundaries in the original input space.