= Selection on an extreme first measurement
{title2=$\mathbb E[Y\mid X>a]=0.6\phi(a)/(1-\Phi(a))$}
Select units with first reading $X>1$, keep that membership fixed, and measure the same units again. The first reading now has a <truncated normal distribution>. For a general finite threshold $a$, write $\phi$ for the <standard normal density> and $\Phi$ for the <standard normal distribution function>. Since $\phi'(x)=-x\phi(x)$,
$$
\mathbb E[X\mid X>a]
=\frac{\int_a^\infty x\phi(x)\,dx}{1-\Phi(a)}
=\frac{\phi(a)}{1-\Phi(a)}.
$$
The <tower property> of <conditional expectation> lets us average the line $\mathbb E[Y\mid X]=0.6X$ over the selected group.<Probability: Theory and Examples (fifth edition) by Rick Durrett/4.1 Conditional Expectation>[Theorem 4.1.13(ii), printed p. 212: nested conditioning]{ref} At $a=1$ this gives
$$
\begin{aligned}
\mathbb P(X>1)&\approx0.158655,\\
\mathbb E[X\mid X>1]&\approx1.525135,\\
\mathbb E[Y\mid X>1]&=0.6\mathbb E[X\mid X>1]\approx0.915081,\\
\mathbb E[Y-X\mid X>1]&\approx-0.610054.
\end{aligned}
$$
The average second reading is below the selection threshold even though every selected first reading exceeded it. This is a group <expectation>; any finite generated group's <sample mean> fluctuates around it. Selecting again on $Y>1$ would define a different group and answer a different question.
The <latent variable> decomposition makes the apparent drop especially clear. Gaussian conditioning gives $\mathbb E[T\mid X=x]=0.6x$ and $\mathbb E[\varepsilon_1\mid X=x]=0.4x$. Thus
$$
\mathbb E[T\mid X>1]\approx0.915081,\qquad
\mathbb E[\varepsilon_1\mid X>1]\approx0.610054,\qquad
\mathbb E[\varepsilon_2\mid X>1]=0.
$$
The first <measurement error> has zero unconditional <mean>, but positive selected <mean>. The second <measurement error> stays centered because it is independent of selection on $X$. The persistent signal $T$ never changed.
\Image[/regression-toward-mean-selected-decomposition.png]
{title=Selection captures an extreme first error while the persistent signal stays fixed}
{description=Original synthetic repeated measurements, not actual measurements. Left: 40 generated units selected by $X>1$; orange trajectories move farther from zero. Right: analytic selected-group <expectations> decompose the first and second readings into persistent <latent variable> $T$ and <measurement error>. Black dots give generated group <sample means> after selecting from 60,000 synthetic units; their small departures from the bar totals reflect sampling variation. All selections use the first reading only.}
Back to article page
