= Statistical learning
Statistical learning constructs predictive rules from data and studies their <prediction error>, <generalization>, and computational fitting. In <supervised learning>, <training data> contain inputs $X$ and a <response variable> $Y$: <regression analysis> predicts a numerical response, while <classification in statistical learning> predicts a label. <Unsupervised learning> instead seeks structure without supplied responses, as in <cluster analysis> and <principal component analysis>. These tasks connect learning to <probability> and <mathematical optimization>. https://web.stanford.edu/~hastie/ISLR2/Slides/Ch2_Statistical_Learning.pdf[James, Witten, Hastie and Tibshirani, ISLR chapter 2 slides]{ref}
For a fitted rule $\widehat f$ and <loss function> $L$, the predictive goal is
$$
R(\widehat f)=\mathbb E\bigl[L(Y^*,\widehat f(X^*))\mid\mathcal D_n\bigr],
$$
where $(X^*,Y^*)$ is a fresh observation, <independent> of the <training data> $\mathcal D_n$, from the intended target <probability distribution>. This <prediction error> fixes the fitted rule and averages over future observations; averaging again over <training data> gives the learning procedure's average performance. Its practical beauty is that a rule compresses past observations into useful future predictions. Its central difficulty is that excellent <training error> need not imply good <generalization>: <cross-validation> supports <model selection>, and a reserved <test set> assesses the resulting choice.
Back to article page
