Risk function

The risk of an estimator is its expected loss as a function of the unknown parameter. Under quadratic loss,

The mean squared error of an estimator $\hat{\theta }$ is

If independent observations have mean $\theta$ and variance ${\theta }^2$, the scaled sample mean hasFor $\theta \ne 0$, it improves on the unbiased sample mean precisely when $(n-1)/(n+1)<k<1$. The optimal scale is $k=n/(n+1)$ and its risk is ${\theta }^2/(n+1)$. At $\theta =0$ all risks vanish and strict improvement is impossible.

The integrated mean squared error is $\mathbb{E}{\int }_D(\hat{m}(x)-m(x){)}^{2}dx$ over a specified domain $D$. The Tonelli theorem permits integration of the pointwise bias-variance decomposition of mean squared error.

Every estimator with finite second moment satisfies

For $X\sim \mathrm{Bin}(n,\theta )$ andthe mean squared error isWhen ${\theta }_0=1/2$, its maximal risk is at $\theta =1/2$ or at an endpoint of $[0,1]$.

An estimator is admissible when no other estimator has risk no larger at every parameter value and strictly smaller at at least one value.

For one observation $X\sim N_p(\theta ,I_p)$ with $p\ge 3$, the James–Stein estimator isUnder quadratic loss, Stein's risk identity givesso it dominates the usual estimator $X$.

The James-Stein risk is strictly below the dimension at every finite parameter, but its improvement tends to zero as the mean norm tends to infinity. Strict pointwise domination therefore leaves the supremum risk unchanged.

A minimax estimator minimizes the supremum of its risk over the parameter space.

For $X\sim \mathrm{Bin}(n,\theta )$, the posterior mean under $\mathrm{Beta}(\sqrt{n}/2,\sqrt{n}/2)$ has constant squared-error risk $1/[4(\sqrt{n}+1{)}^2]$. Its unique Bayes property and constant risk make it the unique minimax estimator.

The minimax risk is the infimum over estimators of the supremum of their risk functions over a parameter class. An upper bound supplies an estimator; a lower bound applies to every estimator, often using the Le Cam two-point lemma.

A risk function can decay rapidly for each fixed regression function while its supremum over a function class decays more slowly or fails to decay. Bounds depending on each function's local differentiability remainder need not be uniform. In fixed-design nonparametric regression, a class with no common smoothness bound permits smooth bump functions whose supports shrink between design points. Such sequences obstruct uniform estimation without contradicting fixed-function mean squared error bounds.

For $X\sim N_p(\theta ,I_p)$ under quadratic loss, the estimator ${\delta }_0(X)=X$ has constant risk $p$ and is minimax. Gaussian priors $N_p(0,c^2{I}_p)$ have Bayes risks $p{c}^2/(1+c^2)$, which tend to $p$ and therefore give the matching minimax lower bound.