Past exam of the mathematics course of the University of Cambridge / 2019 / iii / Paper 218 / 4 / d / Solution

Let $k$ be a positive-definite kernel with feature map $\Phi$ into a reproducing-kernel Hilbert space. Perform regularized LDA on $\Phi (x)$, replacing the within-class covariance operator $C$ by $C+\lambda I$ with $\lambda >0$, which is invertible. The representer property expresses all required inner products and discriminants through the Gram matrix $K_{ij}=k(x_i,x_j)$ and vectors $k(x_i,x)$. This is Kernel LDA. For a nonlinear kernel such as the Gaussian radial-basis kernel, its affine boundaries in feature space pull back to nonlinear boundaries in the original input space.