Mean squared error ($\mathrm{MSE}$)

The mean squared error of an estimator $\hat{\theta }$ is

The asymptotic mean squared error is the leading large-sample approximation to the sum of variance and squared bias of an estimator. Minimizing it balances stochastic variation against systematic error; the resulting choice must still satisfy the assumptions used in the approximation.

If independent observations have mean $\theta$ and variance ${\theta }^2$, the scaled sample mean hasFor $\theta \ne 0$, it improves on the unbiased sample mean precisely when $(n-1)/(n+1)<k<1$. The optimal scale is $k=n/(n+1)$ and its risk is ${\theta }^2/(n+1)$. At $\theta =0$ all risks vanish and strict improvement is impossible.

The integrated mean squared error is $\mathbb{E}{\int }_D(\hat{m}(x)-m(x){)}^{2}dx$ over a specified domain $D$. The Tonelli theorem permits integration of the pointwise bias-variance decomposition of mean squared error.

For a second-order kernel density estimator, write $R(q)=\int q^2$ and ${\mu }_2(K)=\int u^{2}K(u)du$. Under the usual smoothness and shrinking-bandwidth assumptions, the leading mean integrated squared error is the displayed sum of integrated variance and squared bias of an estimator. Its minimizer is $h=[R(K)/(n{\mu }_2(K{)}^{2}R(f^{\prime \prime })){]}^{1/5}$, giving order $n^{-4/5}$. An added $O(n^{-1})$ integrated covariance term is lower order at this bandwidth and does not change its leading constant.

Every estimator with finite second moment satisfies

For $X\sim \mathrm{Bin}(n,\theta )$ andthe mean squared error isWhen ${\theta }_0=1/2$, its maximal risk is at $\theta =1/2$ or at an endpoint of $[0,1]$.