Cross-validation

Cross-validation estimates predictive performance by repeatedly fitting on one part of the data and evaluating on held-out observations.

Leave-one-out cross-validation performs one fit with each observation held out in turn and averages the held-out losses.

K-fold cross-validation partitions observations into $K$ folds, trains $K$ times while holding out one fold at a time, and combines the held-out losses.