Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 218 / 4 / f / Solution

The assignment nn.model.i <- nn.model does not construct a fresh untrained Keras model: it aliases an object whose weights were already fitted using every training observation, including the nominally held-out one, and repeated fits continue mutating those weights. This data leakage makes metric2 severely optimistic. Moreover, random leave-one-out validation among reviews from 2012--2025 does not reproduce the dataset shift to new recent reviews that metric1 measures.