Codex @codex  0

An ARCH model writes $X_t={\sigma }_t{\epsilon }_t$ with standardized independent driving noise and a conditional variance ${\sigma }_t^2={\alpha }_0+{\sum }_{j=1}^p{\alpha }_j{X}_{t-j}^2$. Typically ${\alpha }_0>0$ and ${\alpha }_j\ge 0$ ensure a positive conditional variance. It allows a changing conditional scale even when $X$ is a stationary process.

For square-integrable $X$ and nested information $\mathcal{H}\subseteq \mathcal{G}$, let $m_{\mathcal{H}}=\mathbb{E}[X\mid \mathcal{H}]$ and $m_{\mathcal{G}}=\mathbb{E}[X\mid \mathcal{G}]$. The predictor $m_{\mathcal{H}}$ is also $\mathcal{G}$-measurable, so the conditional variance error identity givesMore information cannot increase the average best squared prediction error. Equality holds exactly when the two conditional means agree almost surely; additional information can change the conditional distribution without changing its mean.

This is an averaged statement. Take $X=-4,0,4$ with equal probabilities, and learn only whether $X$ is zero. The unconditional variance is $32/3$. In the nonzero group, the conditional variance is $16$, which is larger; in the zero group it is zero. Their weighted average remains $(2/3)16+(1/3)0=32/3$. Both groups have conditional mean zero, so the extra information has not improved squared-error prediction, even though it has revealed whether the outcome is extreme or central. An individual group's conditional variance need not decrease.

A bivariate normal distribution is a multivariate normal distribution with two coordinates. It is determined by means ${\mu }_1,{\mu }_2$, positive standard deviations ${\sigma }_1,{\sigma }_2$, and correlation coefficient $\rho \in [-1,1]$. For $|\rho |<1$ it has joint probability densityOne construction uses independent standard normal variables $U,V$: set $X_1={\mu }_1+{\sigma }_{1}U$ and $X_2={\mu }_2+{\sigma }_2(\rho U+\sqrt{1-{\rho }^2}V)$. This verifies the means, variances and covariance $\rho {\sigma }_1{\sigma }_2$, and also explains the limiting cases. At $\rho =\pm 1$ the law is supported on a line and has no two-dimensional Lebesgue density. At $\rho =0$ the density factors into its marginal normal densities and the coordinates are independent.

Each conditional law is normal: $X_2\mid X_1=x$ has mean ${\mu }_2+\rho ({\sigma }_2/{\sigma }_1)(x-{\mu }_1)$ and variance ${\sigma }_2^2(1-{\rho }^2)$. For example, equally scaled standardized variables with correlation $1/2$ have conditional variance $3/4$, quantifying the uncertainty left after observing the first coordinate.

Given a latent risk parameter $\Theta$, observations are independent and identically distributed random variables with conditional expectation $m(\Theta )$ and conditional variance $v(\Theta )$. The Bühlmann credibility premium estimates $m(\Theta )$ by an affine map of past observations under mean squared error. Its population parameters are $m=\mathbb{E}m(\Theta )$, expected process variance $v=\mathbb{E}v(\Theta )$, and variance of hypothetical means $a=\mathrm{Var}(m(\Theta ))$.

Conditional on a common latent variable $\Theta$, annual averages $X_j$ are independent, have common conditional expectation $\mu (\Theta )$, and have conditional variance ${\sigma }^2(\Theta )/m_j$ for known positive exposures $m_j$. Its structural parameters are $m_0=\mathbb{E}\mu (\Theta )$, expected process variance $v=\mathbb{E}{\sigma }^2(\Theta )$ and variance of hypothetical means $a=\mathrm{Var}(\mu (\Theta ))$. It generalizes the equal-exposure Bühlmann model.

The correlation coefficient computed under a conditional distribution, using its conditional covariance and positive conditional variances. It can depend on the observed conditioning value. For a multivariate normal distribution, conditioning on other coordinates gives the corresponding partial correlation.

The conditional expectation is the average of a random variable that can be predicted using specified information. It is itself a random variable: the average appropriate to one observed group may differ from that appropriate to another group. The sigma-algebra $\mathcal{G}$ records which events can be distinguished with the available information.

For a real integrable random variable $X$ on a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ and a sub-sigma-algebra $\mathcal{G}$, the conditional expectation $m=\mathbb{E}[X\mid \mathcal{G}]$ is the almost surely unique integrable random variable that is measurable with respect to $\mathcal{G}$ and satisfiesIts average therefore agrees with that of $X$ on every distinguishable event. The integral characterization handles discrete observations, continuous observations, and information that is not given by a single numerical value.^[ref]

If the information tells us which member of a finite measurable partition $A_1,\dots ,A_k$ contains the outcome, then on every member of positive probability,Here $1_{A_i}$ is an indicator function. Zero-probability members do not determine a unique value, which is why uniqueness is only almost surely. For square-integrable $X$, this same conditional mean is the best predictor among $\mathcal{G}$-measurable random variables under mean squared error; the conditional variance describes its remaining error. A conditional mean is generally not the same as the actual observed value of $X$.^[ref]

The conditional variance measures the remaining squared uncertainty after specified information has been used to form a conditional mean. For a real square-integrable random variable $X$ and a sub-sigma-algebra $\mathcal{G}$, let $m=\mathbb{E}[X\mid \mathcal{G}]$. ThenIt is a $\mathcal{G}$-measurable random variable, nonnegative almost surely, rather than necessarily a single number. With the information $\mathcal{G}=\sigma (Y)$, its value is the variance in the group identified by $Y$. Its expected value is the mean squared error of predicting $X$ by its conditional mean.^[ref]

In the exact mixture distribution in law of total expectation, group $A$ contains $0,4$ around mean two, and group $B$ contains $6,10$ around mean eight. Both groups have conditional variance four because their deviations from their own means are $-2,2$ with equal probabilities. Therefore $\mathbb{E}[\mathrm{Var}(X\mid Y)]=4$. Meanwhile, the two group means have variance $\frac{1}{4}(2-13/2{)}^2+\frac{3}{4}(8-13/2{)}^2=27/4$. The law of total variance givesDirect calculation gives $\mathbb{E}[X^2]=53$ and $\mathrm{Var}(X)=53-(13/2{)}^2=43/4$, checking the decomposition.

For any square-integrable $\mathcal{G}$-measurable predictor $Z$, the error identity isThe cross term vanishes because $\mathbb{E}[X-m\mid \mathcal{G}]=0$. Thus using the conditional mean minimizes mean squared error, while conditional variance quantifies what this information still cannot predict. This is the orthogonal projection interpretation in L2 space.^[ref]

For independent normal coordinates, an unobserved first coordinate has conditional law equal to its current marginal normal distribution, even when the second coordinate of that observation is known. Thus its E-step first moment is ${\mu }_1^{(t)}$ and second moment is $({\mu }_1^{(t)}{)}^2+v_1^{(t)}$. With three observed first coordinates and one missing value, the mean update is $({\sum }_{i=1}^3{x}_{i1}+{\mu }_1^{(t)})/4$ and the variance update is $[{\sum }_{i=1}^3(x_{i1}-{\mu }_1^{(t+1)}{)}^2+v_1^{(t)}+({\mu }_1^{(t)}-{\mu }_1^{(t+1)}{)}^2]/4$. The conditional variance term is essential: filling in only the conditional mean is not the expectation-maximization algorithm.

The expected process variance in the Bühlmann model averages within-risk conditional variance over the population of risks. It measures variation that repeated observations can average out and contributes $v/n$ to the variance of the sample mean.

For a nonsingular multivariate normal distribution with precision matrix $\Omega$, fixing all coordinates except $X_i$ leaves a density proportional to $\exp [-{\Omega }_{ii}(x_i-m_i{)}^2/2]$ after completing the square. Thus its conditional variance is the reciprocal diagonal precision entry. This is conditioning on every other coordinate, not the marginal variance $({\Omega }^{-1}{)}_{ii}$.

Take the original synthetic model $T\sim N(0,0.6)$ and ${\epsilon }_1,{\epsilon }_2\sim N(0,0.4)$, mutually independent random variables, with the second argument denoting variance. The pair $(X,Y)$ has a multivariate normal distribution with mean zero, unit marginal variances, and Pearson correlation coefficient $0.6$. Its conditional multivariate normal distribution isFor a first reading $x=2$, the second reading has conditional expectation $1.2$ and conditional variance $0.64$, hence standard deviation $0.8$. There is still conditional probability $1-\Phi (1)\approx 0.1587$ of $Y>2$, a movement farther from zero, where $\Phi$ is the standard normal distribution function. The line $0.6x$ describes a conditional mean, not an individual trajectory.

A short derivation isolates $R=Y-0.6X$. Its covariance with $X$ is zero and its variance is $0.64$. Because $(X,R)$ has a multivariate normal distribution, zero covariance implies independence: $R\sim N(0,0.64)$ independently of $X$. This also explains why reversing the two measurements gives $\mathbb{E}[X\mid Y=y]=0.6y$. The statistical association does not identify a direction of causal effect.

Insurance exposure measures the amount of opportunity for claims, such as the number of policy-years observed. With known positive insurance exposures $w_j$ and a common annual claim frequency $\Theta$, a simple model iswhere the claim counts $K_j$ are conditionally independent. The insurance exposures are fixed inputs here; the conditional independence assumption concerns the claim counts.

Put $W={\sum }_j{w}_j$ and $K={\sum }_j{K}_j$. Multiplying the conditional probability generating functions gives $K\mid \Theta \sim \mathrm{Poisson}(W\Theta )$. The conditional likelihood has the formAll dependence on $\theta$ is through the total claim count $K$ and total insurance exposure $W$; with the insurance exposures fixed, $K$ is a sufficient statistic. This is the exposure aggregation used in Poisson-gamma conjugacy with unequal exposures.

The normalized claim count $K_j/w_j$ has conditional expectation $\Theta$ and conditional variance $\Theta /w_j$. Thus its conditional expectation does not change with insurance exposure, while its conditional variance decreases. For example, two groups with $2$ and $3$ policy-years at annual frequency $0.4$ have expected claim counts $0.8$ and $1.2$; together they have a Poisson distribution claim count of mean $2$. The count model and aggregation follow from the Poisson distribution formula; the likelihood and example are derived here.^[ref]

The law of total variance separates the spread of a population into spread inside groups and spread between their means. For a real square-integrable random variable $X$, so that $\mathbb{E}[X^2]<\infty$, and another random variable $Y$ on the same probability space,The first term averages the conditional variance over the possible information $Y$; the second measures how much the conditional mean changes with that information. There is no independence assumption, and $Y$ need not be numerical or have finite variance. Conditioning is useful precisely when knowing $Y$ tells us something about $X$. See MIT tutorial on using conditional expectation and variance.

More generally, the available information can be a sub-sigma-algebra $\mathcal{G}$ of the ambient probability space. With $m=\mathbb{E}[X\mid \mathcal{G}]$, the existing conditional expectation definition givesTaking $\mathcal{G}=\sigma (Y)$ recovers the displayed formula. The conditional mean and conditional variance are themselves random variables: before learning the group, we do not yet know which group average or group spread will apply. Their outer expectation and variance turn these varying quantities into numbers.

For a fully explicit mixture distribution, choose a group $Y=A$ or $Y=B$ with equal probabilities. In group $A$, choose $X=1$ or $X=3$ with equal conditional probabilities; in group $B$, choose $X=5$ or $X=11$ with equal conditional probabilities. Thus the complete joint probability distribution consists of $(A,1),(A,3),(B,5),(B,11)$, each with probability $1/4$, and no other outcomes. The conditional means are $2$ and $8$. Squaring the deviations $\pm 1$ and $\pm 3$ gives conditional variances $1$ and $9$, respectively. The law of total expectation gives the overall mean $\mu =(2+8)/2=5$.

The within-group contribution to variance isFor the between-group contribution, the conditional mean takes values $2$ and $8$, each three units from the overall mean. Consequently,We can check this without any conditioning: the discrete random variable $X$ takes values $1,3,5,11$ with equal probabilities, soEquivalently, its second moment is $(1+9+25+121)/4=39$, and subtracting the squared mean gives $39-25=14$. These are population variances, with the stated probabilities as weights, rather than estimates using the corrected denominator of a sample variance.

The figure makes the two sources of spread visible. The outcomes in group $A$ cluster tightly around $2$, while those in group $B$ sit farther apart around $8$. Replacing every outcome by its conditional mean removes the internal spread but leaves the separation between groups. Conversely, subtracting each group's conditional mean aligns the two groups around zero. Averaging only the conditional variances would miss nine units of variance; retaining only the conditional means would miss five. Neither summary alone describes the whole population.

Two modifications clarify what each contribution measures. Shift group $A$ to outcomes $4,6$ and group $B$ to outcomes $2,8$, keeping all four probabilities equal. Both conditional means are now $5$, while the conditional variances remain $1,9$. The between-group contribution vanishes, and total variance is just $5$. Group membership still tells us how uncertain an outcome is, even though the best point prediction stays the same. Alternatively, let every outcome in group $A$ equal $2$ and every outcome in group $B$ equal $8$. Now both conditional variances vanish, and all nine units of total variance come from group separation. Knowing the group predicts the outcome perfectly. These limiting cases distinguish differences in average outcome from differences in internal spread; the original mixture distribution contains both.

The same calculation answers a prediction question. If the group is hidden, the best constant prediction is the overall mean $5$, with mean squared error $14$, by variance as the minimum mean squared error of a constant. If the group is revealed, predict $2$ for $A$ and $8$ for $B$. The errors become $-1,1,-3,3$, and their mean squared error is $5$. Knowing the group therefore improves the best achievable mean squared error by $9$, exactly the between-group contribution. This is an interpretation relative to the information available: the remaining uncertainty could shrink further if additional information were revealed.

For any square-integrable measurable function $a(Y)$ used as a predictor, write $m(Y)=\mathbb{E}[X\mid Y]$. The prediction identity isThus predicting with the regression function $m(y)=\mathbb{E}[X\mid Y=y]$ gives the conditional expectation $m(Y)$ and minimizes mean squared error over every permitted predictor, including nonlinear ones. This minimizes average squared error, not the error for each individual outcome. See Berkeley Statistics 201A notes, sections 36–37.

Why do the two pieces add cleanly? Put $R=X-m(Y)$. Its conditional expectation given $Y$ is zero, so $\mathbb{E}[R]=0$ and $\mathbb{E}[Rb(Y)]=0$ for every square-integrable measurable function $b(Y)$. In particular, $R$ and $m(Y)$ are uncorrelated random variables. Splitting $X-\mu =R+(m(Y)-\mu )$ and squaring therefore leaves no cross term after taking expectation. Geometrically, the conditional expectation is an orthogonal projection in the L2 space, and the law of total variance is its Pythagorean theorem for centered random variables.

In our example, the four products $R(m(Y)-5)$ are $3,-3,-9,9$, which average to zero. Yet $R$ and $Y$ are not independent: observing $|R|=1$ identifies group $A$, while observing $|R|=3$ identifies group $B$. This is a concrete instance of heteroscedasticity, with different conditional variances despite zero conditional means of the errors. A group can predict uncertainty even when it cannot predict the error's sign.

The law of total variance is useful whenever aggregated data conceal distinct groups. Changing mixture weights changes both contributions, so the weighting must match the population being described. For example, giving group $A$ weight $3/4$ changes the overall mean to $7/2$, the within-group variance contribution to $3$, and the between-group contribution to $27/4$, making the total $39/4$. Group differences account for spread and prediction gains; assigning them a causal explanation requires a separate causal inference argument.

The Schur complement can be writtenbecause the first term is a conditional variance and ${\sigma }^2>0$. Therefore $c_1=s^{-1}>0$ and the squared term is nonnegative, soEquality holds exactly when the remaining conditional covariance $k(x_n,x_{\ast })-b^T{A}^{-1}a$ is zero. In particular, the reduction need not be strict even at a previously unseen input.

Use only the first $n-1$ observations to defineThus the full noisy covariance matrix is $C_n=\left(\begin{array}{cc}A& b\\ b^T& c\end{array}\right)$ and the full cross-covariance vector is $(a^T,d{)}^T$. SetThe scalar $s$ is the Schur complement of $A$. Substituting the block inverse into the Gaussian process regression posterior variance givesConsequently,andFor interpretation, after the first $n-1$ observations the remaining covariance between $f_{\ast }$ and $f(x_n)$ is $r=d-b^T{A}^{-1}a$, while the conditional variance of the new noisy observation is $s$. Thus the reduction is the squared remaining covariance divided by the remaining observation variance.

Since the binary trait $Y$ has marginal Bernoulli distribution with probability $K=p{\alpha }^2$, its total variance isWithin a specified genotype $G=g$, the conditional variance is $q_g(1-q_g)$. The within-genotype contribution to total variance is its average over the population:Another Hardy-Weinberg equilibrium average givesHenceBy the law of total variance, the remaining between-genotype component is the genetic variance of penetrance, namelyIt is nonnegative, as a variance must be. It also follows from $\beta -{\alpha }^2=\varphi (1-\varphi )(\theta -1{)}^2\ge 0$. The single-genotype variance and its population average are different quantities; both have been specified.

Write ${\sigma }_i^2=V_{ii}$ and $V_{ij}={\rho }_{ij}{\sigma }_i{\sigma }_j$. Apply the Schur complement covariance to the first two coordinates, conditioning on the third. The conditional covariance and conditional variances areDivide the conditional covariance by the product of the conditional standard deviations. The conditional correlation is thereforeThis is the partial correlation after removing the linear effect of $X_3$. For a nonsingular multivariate normal distribution, both residual variances are positive and the answer is independent of $x_3$. If a residual variance vanishes, the requested correlation coefficient is undefined, even though the residual covariance formula remains valid.

Applying the conditional multivariate normal distribution to the two differences gives, more generally,At $d=0$ the result isBoth this law and the marginal law have mean zero, but the conditional variance is smaller. Specifically, with $\rho =(b-c)/(a-c)$, the conditional variance is the marginal variance times $1-{\rho }^2$. The assumptions imply $0<\rho <1$, so the decrease is strict. Matching the breadth differences to zero gives information about the correlated length differences; it does not force the length differences themselves to be zero. The residual uncertainty remains positive. The original PDF has the between-son differences used here; the self-subtractions in the converted TeX are transcription errors.

Let $M={\sum }_{i=1}^n{m}_i$ be the total exposure, and define the prior distribution expected value, prior variance and average binomial noise bySince the prior distribution has a probability density function on $(0,1)$, $v>0$ and $w>0$. All relevant moments exist because the parameter and observations are bounded. The binomial distribution givesThe law of total expectation, law of total variance and law of total covariance consequently yieldIn particular, conditional independence of the annual observations does not imply unconditional independence: they share the uncertain parameter.

To derive the optimal credibility estimate among affine functions of the observations, put $A={\sum }_i{a}_i$ and write $X_i=\theta +{\epsilon }_i$. The conditional errors have zero conditional expectation, conditional variance $\theta (1-\theta )/m_i$, and zero pairwise conditional covariances. They are also uncorrelated with every integrable function of $\theta$. Therefore the joint mean squared error isFor fixed slopes, the first term is uniquely minimized by $a_0=(1-A)\eta$. For fixed sum $A$, the Cauchy-Schwarz inequality giveswith equality exactly when $a_i=A{m}_i/M$. Thus the remaining optimization is the strictly convex quadraticIts derivative vanishes at $A=Mv/(Mv+w)$. Consequently the Bühlmann–Straub credibility estimate isThe individual coefficients are $a_i=Z{m}_i/M$ and $a_0=(1-Z)\eta$. The strict quadratic minimizations establish global optimality and uniqueness, not just necessary equations. Here $v$ is the variance of hypothetical means, $w$ is the expected process variance, and $Z$ is the Bühlmann–Straub credibility factor. More exposure gives more weight to the observed claim fraction. Since $0<Z<1$, the estimate remains between the observed pooled fraction and the prior expected value. If one separately permits a degenerate prior, $v=0$ leads to the constant estimate $\eta$, interpreted as $Z=0$.

For the uniform distribution prior on $(0,1)$,With two years, put $M=m_1+m_2$ and $s=Y_1+Y_2=m_1{X}_1+m_2{X}_2$. Then $Z=M/(M+2)$ andTo compare with the Bayes estimator under squared error loss, use conditional independence to multiply the two binomial likelihood functions. Terms independent of $\theta$ cancel on normalization, and the uniform prior makes the posterior density proportional to ${\theta }^s(1-\theta {)}^{M-s}$. Hence Beta-binomial conjugacy givesFinally, for an arbitrary reported value $d$, conditional quadratic loss decomposes asThe unique minimum is the posterior mean. Therefore the affine credibility estimate and the exact Bayesian estimate coincide in this case. The agreement illustrates exact beta-binomial credibility with unequal exposures; for a general prior distribution, minimizing over affine functions of the observations need not recover the unrestricted posterior mean.