Stein's lemma (probability) (source code)

= Stein's lemma
{disambiguate=probability}
{c}
{wiki=Stein's_lemma}

= Gaussian Stein identity
{c}
{synonym}

= Gaussian integration by parts
{c}
{synonym}

= Stein's lemma
{c}
{synonym}

<Stein's lemma> relates Gaussian multiplication to differentiation. Let $Z\sim N(0,1)$. One sufficient set of conditions is that $g$ is locally <absolute continuity of measures>[absolutely continuous], both $\mathbb E|Zg(Z)|$ and $\mathbb E|g'(Z)|$ are finite, and $g(x)\phi(x)\to0$ as $x\to\pm\infty$, where $\phi$ is the <standard normal density>. Then
$$
\mathbb E[Zg(Z)]=\mathbb E[g'(Z)].
$$
Since $\phi'(x)=-x\phi(x)$, <integration by parts> on a finite <interval> moves the <derivative> from $\phi$ to $g$; the boundary condition and integrability justify the infinite-interval limit.

For $g(z)=z$, the identity gives $\mathbb EZ^2=1$. For $g(z)=z^3$, it gives $\mathbb EZ^4=3\mathbb EZ^2=3$. More generally, for $X\sim N(\mu,\sigma^2)$ with $\sigma>0$, <standardization of a normal random variable>[standardization] yields
$$
\mathbb E[(X-\mu)g(X)]=\sigma^2\mathbb E[g'(X)]
$$
under the corresponding conditions. The factor $\sigma^2$ is essential.

This integration-by-parts rule converts <Gaussian error> cross terms into <derivatives> of an <estimator>, which is the mechanism of the <Stein risk identity for normal mean estimators>. Boundary behavior matters for singular <estimators>: formal differentiation alone does not justify applying the identity across a nonintegrable singularity.