Risk function

The risk of an estimator is its expected loss as a function of the unknown parameter. Under quadratic loss,

The mean squared error of an estimator $\hat{\theta }$ is

The asymptotic mean squared error is the leading large-sample approximation to the sum of variance and squared bias of an estimator. Minimizing it balances stochastic variation against systematic error; the resulting choice must still satisfy the assumptions used in the approximation.

If independent observations have mean $\theta$ and variance ${\theta }^2$, the scaled sample mean hasFor $\theta \ne 0$, it improves on the unbiased sample mean precisely when $(n-1)/(n+1)<k<1$. The optimal scale is $k=n/(n+1)$ and its risk is ${\theta }^2/(n+1)$. At $\theta =0$ all risks vanish and strict improvement is impossible.

The integrated mean squared error is $\mathbb{E}{\int }_D(\hat{m}(x)-m(x){)}^{2}dx$ over a specified domain $D$. The Tonelli theorem permits integration of the pointwise bias-variance decomposition of mean squared error.

For a second-order kernel density estimator, write $R(q)=\int q^2$ and ${\mu }_2(K)=\int u^{2}K(u)du$. Under the usual smoothness and shrinking-bandwidth assumptions, the leading mean integrated squared error is the displayed sum of integrated variance and squared bias of an estimator. Its minimizer is $h=[R(K)/(n{\mu }_2(K{)}^{2}R(f^{\prime \prime })){]}^{1/5}$, giving order $n^{-4/5}$. An added $O(n^{-1})$ integrated covariance term is lower order at this bandwidth and does not change its leading constant.

Every estimator with finite second moment satisfies

For $X\sim \mathrm{Bin}(n,\theta )$ andthe mean squared error isWhen ${\theta }_0=1/2$, its maximal risk is at $\theta =1/2$ or at an endpoint of $[0,1]$.

A decision rule is admissible if no other rule has risk function no greater at every parameter and strictly smaller at one parameter. Parameter estimation is one decision problem; its admissible rules are admissible estimators.

An estimator is admissible when no other estimator has risk no larger at every parameter value and strictly smaller at at least one value.

For one observation $X\sim N_p(\theta ,I_p)$ with $p\ge 3$, the James–Stein estimator isUnder quadratic loss, Stein's risk identity givesso it dominates the usual estimator $X$.

For $X\sim N_p(\theta ,I)$ and $p\ge 4$, this estimator preserves the sample mean and applies James–Stein shrinkage in its orthogonal $(p-1)$-dimensional contrast subspace. Its risk under quadratic loss is $p-(p-3{)}^2{\mathbb{E}}_{\theta }\Vert X-\bar{X}{1}_p{\Vert }^{-2}$. The noncentrality is $\Vert \theta -\bar{\theta }{1}_p{\Vert }^2$, so the reciprocal moment of a noncentral chi-squared variable makes the risk minimal exactly when all true components coincide; the minimum is $3$. A negative realized shrinkage factor reverses contrast signs, and its magnitude can exceed one for sufficiently small contrast norm.

The James-Stein risk is strictly below the dimension at every finite parameter, but its improvement tends to zero as the mean norm tends to infinity. Strict pointwise domination therefore leaves the supremum risk unchanged.

A minimax estimator minimizes the supremum of its risk over the parameter space.

For $X\sim \mathrm{Bin}(n,\theta )$, the posterior mean under $\mathrm{Beta}(\sqrt{n}/2,\sqrt{n}/2)$ has constant squared-error risk $1/[4(\sqrt{n}+1{)}^2]$. Its unique Bayes property and constant risk make it the unique minimax estimator.

The minimax risk is the infimum over estimators of the supremum of their risk functions over a parameter class. An upper bound supplies an estimator; a lower bound applies to every estimator, often using the Le Cam two-point lemma.

A risk function can decay rapidly for each fixed regression function while its supremum over a function class decays more slowly or fails to decay. Bounds depending on each function's local differentiability remainder need not be uniform. In fixed-design nonparametric regression, a class with no common smoothness bound permits smooth bump functions whose supports shrink between design points. Such sequences obstruct uniform estimation without contradicting fixed-function mean squared error bounds.

For $X\sim N_p(\theta ,I_p)$ under quadratic loss, the estimator ${\delta }_0(X)=X$ has constant risk $p$ and is minimax. Gaussian priors $N_p(0,c^2{I}_p)$ have Bayes risks $p{c}^2/(1+c^2)$, which tend to $p$ and therefore give the matching minimax lower bound.