 Statistical decision theory

Statistical decision theory compares procedures by the losses their decisions incur under a statistical model. Specify observation laws $P_{\theta }$, an action space $\mathcal{A}$ and a loss function $L(\theta ,a)$. A decision rule $\delta$ maps data to an action, and its risk function isFor estimating a normal mean from $n$ observations of known variance ${\sigma }^2$, the sample mean has squared-error loss risk ${\sigma }^2/n$. Under zero-one loss, the same framework describes classification in statistical learning and statistical hypothesis testing.

Different criteria compare the whole risk function differently. A Bayes decision rule minimizes risk averaged over a prior distribution. A minimax decision rule minimizes the worst-case risk. An admissible decision rule has no competitor with no greater risk everywhere and smaller risk somewhere. These criteria require the model, allowed actions and loss to be specified; accuracy alone does not define what mistakes cost.