Past exam of the mathematics course of the University of Cambridge / 2019 / iii / Paper 218 / 4 / c / Solution

When $p>n$, the pooled within-class covariance has rank at most $n-L<p$, so it is singular and ordinary LDA is not defined. For every $\lambda >0$,is positive definite because $v^T{\hat{\Sigma }}_{\lambda }v\ge \lambda \Vert v{\Vert }^2>0$. Replacing the covariance by this matrix gives well-defined regularized LDA. Choose $\lambda$ for predictive performance by cross-validation.