Charles M. Stein Created 2026-10-18 Updated 2026-11-05
Charles M. Stein changed the theory of estimation by showing that estimating several normal means together can permit improvement over their separate usual estimates under a combined squared-error loss. The James–Stein estimator expresses this phenomenon through shrinkage: introducing bias can lower the total mean squared error when variance is reduced sufficiently. The guarantee concerns the specified aggregate loss, rather than promising a better estimate for every coordinate in every dataset.
Stein also developed tools connecting normal distributions, derivatives and probability approximations. With Bradley Efron, he coauthored The Jackknife Estimate of Variance by Bradley Efron and Charles M. Stein, a source associated with the Efron–Stein inequality. His Stanford obituary records his mathematical career and death in 2016.[ref]
Figure 1.
Portrait of Charles M. Stein
.
For a decision rule and loss , its risk function is . It is an admissible estimator if there is no rule with risk everywhere no larger and somewhere strictly smaller. A Bayes decision rule minimizes for a specified prior . A minimax decision rule minimizes over all rules.
Here the maximum-likelihood estimator is , and its squared-error risk is
Write the proposed estimator as with for , and assign any value at zero, an event of probability zero. Expanding the squared loss gives
Stein's lemma for a unit-covariance normal vector says when the relevant derivatives and expectations are integrable. With ,
The singularity causes no difficulty in three dimensions: is locally integrable against the volume element , and the normal tails control infinity. More formally apply Stein's lemma first to , then let ; its squared norm and derivative magnitudes are bounded by constants times , so dominated convergence justifies the limit. Consequently
The expectation is finite and strictly positive. Thus this James–Stein estimator strictly dominates the maximum likelihood estimator, which is inadmissible under the specified squared-error loss.
The James–Stein estimator in the specified unit-variance model is
with an arbitrary value at , a probability-zero event. The risk of is . Gaussian Stein identity states for suitably integrable differentiable functions; it extends to the present singular function by truncation, since makes finite.
Put and . Then and . Expanding the quadratic risk and applying Stein's lemma yields
Thus domination is strict for every finite .
To identify the worst-case risk, let . On , the reciprocal square is at most . On the complementary ball, the Gaussian density is at most , so its reciprocal-square integral is bounded by a constant times , tending to zero. Therefore , and from below. Hence
The second supremum is approached at infinity and need not be attained at a finite parameter.
An admissible estimator is one for which there is no estimator satisfying
for every , with strict inequality for at least one . For , the James–Stein estimator
has risk
It strictly dominates , and hence