Law of total variance Created 2026-09-24 Updated 2026-10-14
The law of total variance separates the spread of a population into spread inside groups and spread between their means. For a real square-integrable random variable , so that , and another random variable on the same probability space,
The first term averages the conditional variance over the possible information ; the second measures how much the conditional mean changes with that information. There is no independence assumption, and need not be numerical or have finite variance. Conditioning is useful precisely when knowing tells us something about . See MIT tutorial on using conditional expectation and variance.
More generally, the available information can be a sub-sigma-algebra of the ambient probability space. With , the existing conditional expectation definition gives
Taking recovers the displayed formula. The conditional mean and conditional variance are themselves random variables: before learning the group, we do not yet know which group average or group spread will apply. Their outer expectation and variance turn these varying quantities into numbers.
For a fully explicit mixture distribution, choose a group or with equal probabilities. In group , choose or with equal conditional probabilities; in group , choose or with equal conditional probabilities. Thus the complete joint probability distribution consists of , each with probability , and no other outcomes. The conditional means are and . Squaring the deviations and gives conditional variances and , respectively. The law of total expectation gives the overall mean .
The within-group contribution to variance is
For the between-group contribution, the conditional mean takes values and , each three units from the overall mean. Consequently,
We can check this without any conditioning: the discrete random variable takes values with equal probabilities, so
Equivalently, its second moment is , and subtracting the squared mean gives . These are population variances, with the stated probabilities as weights, rather than estimates using the corrected denominator of a sample variance.
Figure 1.
Four equally likely outcomes split total variance into within-group 5 and between-group 9
. Details.
The figure makes the two sources of spread visible. The outcomes in group cluster tightly around , while those in group sit farther apart around . Replacing every outcome by its conditional mean removes the internal spread but leaves the separation between groups. Conversely, subtracting each group's conditional mean aligns the two groups around zero. Averaging only the conditional variances would miss nine units of variance; retaining only the conditional means would miss five. Neither summary alone describes the whole population.
Two modifications clarify what each contribution measures. Shift group to outcomes and group to outcomes , keeping all four probabilities equal. Both conditional means are now , while the conditional variances remain . The between-group contribution vanishes, and total variance is just . Group membership still tells us how uncertain an outcome is, even though the best point prediction stays the same. Alternatively, let every outcome in group equal and every outcome in group equal . Now both conditional variances vanish, and all nine units of total variance come from group separation. Knowing the group predicts the outcome perfectly. These limiting cases distinguish differences in average outcome from differences in internal spread; the original mixture distribution contains both.
The same calculation answers a prediction question. If the group is hidden, the best constant prediction is the overall mean , with mean squared error , by variance as the minimum mean squared error of a constant. If the group is revealed, predict for and for . The errors become , and their mean squared error is . Knowing the group therefore improves the best achievable mean squared error by , exactly the between-group contribution. This is an interpretation relative to the information available: the remaining uncertainty could shrink further if additional information were revealed.
For any square-integrable measurable function used as a predictor, write . The prediction identity is
Thus predicting with the regression function gives the conditional expectation and minimizes mean squared error over every permitted predictor, including nonlinear ones. This minimizes average squared error, not the error for each individual outcome. See Berkeley Statistics 201A notes, sections 36–37.
Why do the two pieces add cleanly? Put . Its conditional expectation given is zero, so and for every square-integrable measurable function . In particular, and are uncorrelated random variables. Splitting and squaring therefore leaves no cross term after taking expectation. Geometrically, the conditional expectation is an orthogonal projection in the L2 space, and the law of total variance is its Pythagorean theorem for centered random variables.
Figure 2.
Conditional prediction leaves zero group means but different residual spreads
. Details.
In our example, the four products are , which average to zero. Yet and are not independent: observing identifies group , while observing identifies group . This is a concrete instance of heteroscedasticity, with different conditional variances despite zero conditional means of the errors. A group can predict uncertainty even when it cannot predict the error's sign.
The law of total variance is useful whenever aggregated data conceal distinct groups. Changing mixture weights changes both contributions, so the weighting must match the population being described. For example, giving group weight changes the overall mean to , the within-group variance contribution to , and the between-group contribution to , making the total . Group differences account for spread and prediction gains; assigning them a causal explanation requires a separate causal inference argument.
Since the binary trait has marginal Bernoulli distribution with probability , its total variance is
Within a specified genotype , the conditional variance is . The within-genotype contribution to total variance is its average over the population:
Another Hardy-Weinberg equilibrium average gives
Hence
By the law of total variance, the remaining between-genotype component is the genetic variance of penetrance, namely
It is nonnegative, as a variance must be. It also follows from . The single-genotype variance and its population average are different quantities; both have been specified.
Write and . Apply the Schur complement covariance to the first two coordinates, conditioning on the third. The conditional covariance and conditional variances are
Divide the conditional covariance by the product of the conditional standard deviations. The conditional correlation is therefore
This is the partial correlation after removing the linear effect of . For a nonsingular multivariate normal distribution, both residual variances are positive and the answer is independent of . If a residual variance vanishes, the requested correlation coefficient is undefined, even though the residual covariance formula remains valid.
Applying the conditional multivariate normal distribution to the two differences gives, more generally,
At the result is
Both this law and the marginal law have mean zero, but the conditional variance is smaller. Specifically, with , the conditional variance is the marginal variance times . The assumptions imply , so the decrease is strict. Matching the breadth differences to zero gives information about the correlated length differences; it does not force the length differences themselves to be zero. The residual uncertainty remains positive. The original PDF has the between-son differences used here; the self-subtractions in the converted TeX are transcription errors.
Let be the total exposure, and define the prior distribution expected value, prior variance and average binomial noise by
Since the prior distribution has a probability density function on , and . All relevant moments exist because the parameter and observations are bounded. The binomial distribution gives
The law of total expectation, law of total variance and law of total covariance consequently yield
In particular, conditional independence of the annual observations does not imply unconditional independence: they share the uncertain parameter.
To derive the optimal credibility estimate among affine functions of the observations, put and write . The conditional errors have zero conditional expectation, conditional variance , and zero pairwise conditional covariances. They are also uncorrelated with every integrable function of . Therefore the joint mean squared error is
For fixed slopes, the first term is uniquely minimized by . For fixed sum , the Cauchy-Schwarz inequality gives
with equality exactly when . Thus the remaining optimization is the strictly convex quadratic
Its derivative vanishes at . Consequently the Bühlmann–Straub credibility estimate is
The individual coefficients are and . The strict quadratic minimizations establish global optimality and uniqueness, not just necessary equations. Here is the variance of hypothetical means, is the expected process variance, and is the Bühlmann–Straub credibility factor. More exposure gives more weight to the observed claim fraction. Since , the estimate remains between the observed pooled fraction and the prior expected value. If one separately permits a degenerate prior, leads to the constant estimate , interpreted as .
For the uniform distribution prior on ,
With two years, put and . Then and
To compare with the Bayes estimator under squared error loss, use conditional independence to multiply the two binomial likelihood functions. Terms independent of cancel on normalization, and the uniform prior makes the posterior density proportional to . Hence Beta-binomial conjugacy gives
Finally, for an arbitrary reported value , conditional quadratic loss decomposes as
The unique minimum is the posterior mean. Therefore the affine credibility estimate and the exact Bayesian estimate coincide in this case. The agreement illustrates exact beta-binomial credibility with unequal exposures; for a general prior distribution, minimizing over affine functions of the observations need not recover the unrestricted posterior mean.
The Nadaraya–Watson estimator is
For the unit-width box kernel, it averages the responses with design points in . Define it to be zero if that window contains no observations; the supplied small-denominator bound must refer to a defined estimator.
Write , and . Set . On ,
This is the absolute error bound for a ratio estimator with a controlled denominator; it does not require the numerator and denominator to be independent.
The conditional expectation definition of the regression function gives
At this interior point, the Taylor theorem expansions are and . Multiplying and using the symmetric interval removes the odd linear term. More precisely,
This cancellation underlies the interior absolute-error rate of the Nadaraya-Watson estimator.
For the variance, independence between observations gives
Conditional on , the second moment is . The bounded conditional variance, local boundedness of , and boundedness of consequently make the expectation at most . Thus . By the Cauchy-Schwarz inequality,
The auxiliary result supplied in the question makes the expected error on equal to . Combining both events, with , proves
Both the squared-bandwidth bias and the inverse-square-root window-count error have this order. The supplied separate consistency result for the design-density estimator is not needed once the stronger small-denominator expectation bound is used.
A credibility estimate blends the risk's own sample mean with a population or prior expected value. In the basic equal-exposure form it is , where the credibility factor is the weight attached to the individual experience. More data ordinarily increase that weight; greater within-risk variation relative to between-risk variation reduces it.
Write for the conditional claim expected value, keeping for the prior-level parameter. Normalization of the probability density function gives
At interior points of its finite domain, differentiation under the integral sign yields
This is an instance of the exponential-family derivative identities for natural parameter . The minus sign is present in the PDF and is lost in the converted TeX. The conditional variance is likewise .
Differentiate the log of the natural conjugate prior:
Integrate on a compact subinterval and then let its endpoints approach those of the parameter interval. The prescribed vanishing of the prior density eliminates the boundary term. Since , monotone convergence theorem justifies the limit of its expectation, and the identity gives
In particular the assumed proper prior and boundary conditions themselves establish this first-moment integrability.
For and , conditional independence makes the likelihood function proportional to . Multiplication by the prior distribution gives the Bayesian posterior
Thus this is a conjugate prior family, with updated parameters
The integration-by-parts calculation for the posterior mean also needs vanishing posterior boundary values. Here that property can be checked, rather than assumed. Let be the essential lower and upper endpoints of the claim support. Every nondegenerate tilted claim law has , so the established prior mean satisfies . Supported observations have ; hence . At a finite parameter endpoint the prior's vanishing forces , because its exponential factor has a finite positive limit; the increased power makes the posterior vanish there too. At , choose with positive base-measure mass below . Then and the unnormalized posterior is at most . At , choose with positive mass above to obtain the corresponding bound . These bounds also ensure posterior propriety at infinite endpoints. This is the endpoint control for a Laplace-family conjugate posterior.
Apply the previous score integration to the Bayesian posterior. The natural conjugate credibility identity is
Thus is the credibility factor. This is an exact posterior mean, rather than an affine approximation to one.
For the specific shape-two gamma distribution, take
Then and . The gamma rate gamma conjugacy prior is
that is, . Its normalizing constant in the notation of the question is . The gamma distribution density vanishes at both zero and infinity for . The posterior has shape and rate , so its conditional-mean posterior mean is again .
For the last calculation, gamma distribution integration gives
The second identity requires the printed . Therefore the expected process variance and variance of hypothetical means, denoted by in this question, are
Consequently , and the Bühlmann credibility factor agrees with the exact Bayesian weight:
This is the exact Bühlmann credibility for gamma claims, expressed using the reciprocal rate as the random scale. The symbols here are the process and between-risk variances, respectively; their roles should not be interchanged when comparing with other notation for the Bühlmann model.
For the probability generating function , differentiation gives
The second derivative is a factorial moment, not the raw second moment.
In the Galton-Watson process, condition on . The next generation is a sum of independent offspring counts, and so its conditional generating function is . Averaging over gives
The conditional mean is and the conditional variance is . Thus iterated expectation gives from one ancestor, and the law of total variance gives , . Iterating yields the branching-process generation moments
For the critical case , the correct limiting value is . If , nonnegative offspring are zero almost surely, so every later generation has zero mean and variance; handle this directly rather than use an ambiguous zero power. At generation zero the mean is one and variance zero.
The branching-process extinction criterion states that extinction probability is the smallest fixed point of in : probabilities of extinction by successive generations are the iterates of from zero, increasing to that fixed point. Here the equation is , factoring as . The two admissible roots are and one, so
The Rao-Blackwell theorem says that if is a sufficient statistic and a finite-variance estimator, then is an estimator with the same expectation and no larger variance. Sufficiency ensures the conditional distribution, and hence this formula as a function of the data, does not involve the unknown parameter. The tower property of conditional expectation gives , while the law of total variance gives
This proves the theorem and shows that equality holds exactly when is already determined by , almost surely. In particular, unbiasedness is preserved.
Assume . Independence gives , so the proposed indicator is unbiased and has variance . The type counts are a sufficient statistic: conditional on them, every ordering of the labels is equally likely. Consequently,
For example, the conditional probability of the ordered first three labels is . This is Rao-Blackwellization of a multinomial probability product.
If all three are positive, there is positive probability that all three counts are positive. On such an event the conditional indicator has probability strictly between zero and one, so its conditional variance is positive. Thus for interior parameter values. If any , then and both estimators have zero variance; the printed strict inequality is impossible at those boundary parameters. The valid unrestricted statement is the non-strict inequality.
Conditional on , a Poisson distribution has both conditional expectation and conditional variance equal to . For the uniform distribution on ,
Thus and the one-year credibility factor is . With the observed count equal to one, the Bühlmann credibility estimate of the next year's conditional claim expected value is
The estimate gives relatively little weight to one year because the expected process variance is six times the variance of hypothetical means.
Write the year- count as a sum of its individual counts, with . The specified conditional independence within that year gives
Dividing by and scaling the conditional variance by therefore gives
This is the Bühlmann–Straub model: larger exposures reduce the process noise in a year's average.
Define the population parameters and total observed exposure by
Here is the expected process variance and the variance of hypothetical means. Assume finite second moments. The law of total variance gives , and conditional independence between years gives for : their only shared variation is the latent conditional mean.
To derive the Bühlmann–Straub credibility estimate, minimize mean squared error among affine estimates of . For coefficients and , the optimal intercept is . Centering at and conditioning on shows that the resulting error is
The cross term vanishes because has conditional mean zero; the conditional noise cross terms vanish by the between-year independence assumption.
For fixed , the Cauchy-Schwarz inequality gives
with equality when . Minimize the remaining quadratic . Its minimizing value is the Bühlmann–Straub credibility factor
The experience term is exposure-weighted:
Since the future conditional mean count is , multiplying the optimal estimate by the known future exposure gives
It is the best affine linear least-squares projection, not a claim that the exact conditional expectation given all data is always affine. If , the conditional mean is a known constant almost surely and one takes ; if , the observations reveal it without process noise and . The ordinary case has .
For fixed-design nonparametric regression, let and . The degree- local polynomial estimator takes the fitted intercept , where
Use a nonnegative regression kernel and an invertible local polynomial Gram matrix to obtain a unique fit. For , solving the single weighted least squares normal equation gives the local constant estimator:
At the interior point , use the standard unit-integral regression kernel convention . Otherwise the leading variance below is ; unlike a kernel density estimator, the local constant estimator itself is unchanged by rescaling .
The local constant estimator is a linear estimator in nonparametric regression with weights . The independent unit-variance errors imply
To control the bias of an estimator, differentiability at this one point is enough. Write
Symmetry makes , and the supplied moment approximation gives . Therefore the linear contribution to the bias of an estimator is . The remaining contribution has absolute value at most , because only receive positive weight. This is the interior first-order bias cancellation for local constant regression, and gives squared bias of an estimator . Combining with the variance proves
For fixed , take . It satisfies both asymptotic smoothing bandwidth conditions, and the displayed remainder, multiplied by , tends to zero for this fixed . Thus
Letting proves the optimized scaled pointwise error tends to zero. The order of these limits matters: no remainder uniform in has been assumed.
For the lower bound over the whole function class, the absence of a uniform bound on derivatives permits a particularly direct argument. It also avoids assuming a normal distribution for errors when the model only supplies their first two moments. Fix and . Choose a smooth bump function on the interval, with , whose support contains no design point except possibly . Such a bump exists because the design is finite; at an endpoint use the restriction of a smooth bump on the real line. Every , , belongs to the differentiability class.
If is not a design point, all observation means are zero for every . The Le Cam two-point lemma applied to has total variation distance zero and gives a lower bound tending to infinity with . In fact, the supremum of the mean squared error is infinite at every such .
If is a design point, only its response depends on . The resulting location family is the single observation ; all remaining observations are independent of and give no extra information. The single-observation location minimax bound is
Here is a proof covering discrete as well as continuous errors. Give the prior distribution , independently of the noise. Put and . Consider an easier experiment in which an oracle also reveals whether , and reveals itself on the complementary event. On the event and , every possible truncated noise value places within . The flat prior distribution then makes the Bayesian posterior noise law exactly its truncated original law. The smallest conditional squared-error loss is its conditional variance , attained by the posterior mean.
The probability of that event is at least for , by restricting further to . Thus every rule's supremum risk function is at least the original Bayes risk, which is at least the easier experiment's Bayes risk, and therefore at least . Let and then . Finite second moment and zero mean give . The rule has constant mean squared error one, proving the equality.
Consequently, at every , the unrestricted differentiability class satisfies the stronger conclusion
Thus works for every . There is no conflict with the preceding fixed-function limit: the worst functions, including arbitrarily narrow bumps, can vary with . This is the pointwise versus uniform risk distinction. A smoothness ball with a common derivative bound would pose a different minimax risk problem.
Work conditionally on a positive-probability atom of . Let have the conditional distribution of there, and let be an independent copy of that conditional law. The finite sample space makes all expectations finite. Then
Strict increase of makes the integrand nonnegative, and strictly positive whenever . The conditional variance is positive by part (c), so the conditional distribution is nondegenerate and . Thus the numerator of the hedge ratio is strictly positive on every such atom. The denominator is also positive, giving
This is strict positive covariance with an increasing payoff. The independent copy is taken from the conditional distribution, not from an unrelated unconditional distribution.
To distinguish the random intensity from its possible values, write it as . Its law is a gamma distribution with shape and rate . Hence
For the Poisson mixture, conditional expectation and conditional variance both equal . The law of total expectation and the law of total variance yield
where . Applying the random sum of independent claims formulas with the exponential distribution of the claim sizes gives the portfolio B moments
At the matched intensity , the expected value for portfolio A is also , whereas its variance is . Thus the expected totals agree, but mixing increases the variance:
The extra term is precisely .
In the Bühlmann model, a latent risk parameter is drawn from a population distribution. Conditional on , the yearly observations are independent and identically distributed random variables, with conditional expectation and conditional variance . Define the structural parameters
Here is the expected process variance, while is the variance of hypothetical means. The Bühlmann credibility premium is the best affine estimate of from the observed claims, under mean squared error. Predicting the next claim gives the same affine estimate: the extra conditional observation noise contributes the constant to the prediction error.
The law of total variance and conditional independence give
An affine estimate can be written as : for any chosen , optimizing the constant makes its expected value equal to . The normal equations for the linear least-squares projection are
For they force all to agree, with . Thus the credibility factor and premium are
The credibility factor increases with the observation count and between-risk variance, and decreases with within-risk variance. If , the risk mean is known and ; if and , one observation reveals it and . If both vanish, the premium is the fixed value and the factor is immaterial.
In the specified model, the conditional law is a gamma distribution with shape and scale . Therefore
The prior is an inverse-gamma distribution with shape and scale . To obtain its moments directly, substitute in the defining integral, obtaining
The Gamma function recurrence yields
The assumption makes both structural variances finite. Hence
It follows that the model-specific credibility estimate is
For the Bayes estimator under squared error loss, the quantity to estimate is , so the optimum is its posterior mean. The likelihood function, viewed as a function of , is proportional to
Multiplication by the prior shows gamma scale inverse-gamma conjugacy:
Although the printed hint only mentions integer shapes, the same substitution and Gamma integral normalize this posterior for every positive real shape, so no integrality of is needed. Its posterior mean gives the Bayesian estimate and comparison
This exact Bühlmann credibility for gamma claims holds for every observed sample, not merely on average. Here the posterior mean is affine in the sample mean, so the best affine Bühlmann credibility premium is also the unrestricted Bayes estimator under squared error loss.
Two useful features of autoregressive conditional heteroscedasticity are persistent changes in conditional scale and excess unconditional kurtosis. Its time-varying conditional variance can explain volatility clustering, where large absolute returns occur in groups even when signed returns have little autocorrelation. A homoscedastic autoregressive moving-average model has a fixed innovation variance.
Also, a conditional normal distribution with a random scale is a Gaussian scale mixture. Its unconditional kurtosis can exceed , or its fourth moment can be infinite. A Gaussian autoregressive moving-average model remains jointly Gaussian and cannot reproduce this effect. In the particular lag-two ARCH process, persistence of the squared scale occurs within each parity subsequence; the two parity subsequences are independent.
The useful autoregressive model is for the squares, not for signed observations. Put and
Then
The errors form a martingale difference sequence relative to the noise history, because the current standardized noise is independent of the past. When the fourth moment is finite they have finite variance and are uncorrelated across distinct times, although their conditional variance depends on the regressor. In centered form, .
For ordinary least squares, use the response-regressor pairs , , . With their separate means and , minimize . If the regressor sum of squares is positive, the estimators are
The two means use the matched pairs; replacing them indiscriminately by a single full-sample mean is not the exact least-squares formula. The noise-series representation supplies an ergodic stationary process. If , finite regressor second moments and the error's zero conditional expectation justify the usual population regression and statistical consistency argument. The observations still define a finite-sample least-squares fit outside that moment range, but the ordinary finite-variance justification must not be claimed there. If parameter constraints are required, minimize the same criterion subject to and , rather than assert that unconstrained estimates automatically satisfy them.