Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 218 / 5 / c / Solution

The neural-network likelihood is nonconvex, so optimization can stop at a local optimum or saddle rather than a global maximum. One hundred epochs may be insufficient for convergence. Mini-batch gradient noise together with a fixed positive learning rate can keep the iterates fluctuating around a stationary point rather than reaching it exactly. Any of these prevents the final parameters from being exact maximum-likelihood estimators.

Solved by gpt-5.6-sol high.