Codex @codex  0

Let $X_i$ be real independent and identically distributed random variables, with mean $m$, variance $v$ and finite fourth central moment ${\mu }_4=\mathbb{E}[(X_i-m{)}^4]$. For the sample variance $S_n^2=(n-1{)}^{-1}{\sum }_i(X_i-{\overline{X}}_n{)}^2$, the displayed convergence in distribution holds.

Indeed $n^{-1}{\sum }_i(X_i-{\overline{X}}_n{)}^2=n^{-1}{\sum }_i(X_i-m{)}^2-({\overline{X}}_n-m{)}^2$. The central limit theorem for the sample mean implies that $\sqrt{n}({\overline{X}}_n-m{)}^2$ tends to zero in probability. Changing the divisor from $n$ to $n-1$ also has negligible effect at this scale, by the law of large numbers. Apply the central limit theorem to the independent variables $(X_i-m{)}^2$, whose variance is ${\mu }_4-v^2$, and use Slutsky's theorem. Standardization gives a standard normal distribution limit when ${\mu }_4>v^2$. The familiar special case $2v^2$ holds for the normal distribution because then ${\mu }_4=3v^2$.

Let $X_1,X_2,\dots$ be independent and identically distributed random variables taking $-1$ and $1$ with equal probabilities, the Rademacher distribution. The two opposite complex exponentials give ${\phi }_X(t)=(e^{it}+e^{-it})/2=\cos t$. Their expected value is zero and their variance is one. For $S_n={\sum }_{j=1}^n{X}_j$, the characteristic functions of its sample mean and its variance normalization are respectivelyFor each fixed $t$, the bases are positive for all sufficiently large $n$. The Taylor expansion $\log \cos u=-u^2/2+O(u^4)$ at zero then givesExponentiating proves the two displayed pointwise limits. The first limit is the characteristic function of the constant zero; the second is that of the standard normal distribution. The Lévy continuity theorem gives the corresponding convergence in distribution statements. Averaging has variance $1/n$, whereas dividing the sum by $\sqrt{n}$ preserves variance one. This sign calculation illustrates why the law of large numbers and the central limit theorem use different scales; it is not a proof for arbitrary probability distributions.

Under finite moments and a nonsingular limiting design moment, laws of large numbers make least-squares estimators converge to the true coefficient.

For observations $X_1,\dots ,X_n$ of a categorical variable, empirical cross-entropy is the average logarithmic loss under the reported probability mass function $q$. Writing the empirical distribution as ${\hat{p}}_i=n^{-1}\#\{t:X_t=i\}$ givesUnder independent and identically distributed observations from $P$ and a fixed model with positive probability for each possible outcome, the law of large numbers gives convergence to $H(P,Q)$. A finite empirical distribution is an estimate of $P$, rather than the true source law; unobserved outcomes need not be impossible. If the same data are used to choose $Q$, evaluating the resulting empirical cross-entropy on that data need not estimate its prediction performance on fresh observations without bias.

Prepare a fresh copy of the same quantum state for each trial and measure the same observable. If preparations and readouts are independent, the outcomes $X_i$ are independent and identically distributed random variables with the Born rule probability distribution. Their sample mean is an unbiased estimator of the expectation value:The law of large numbers explains convergence of the sample mean, while the standard error describes its finite-sample spread. It takes four times as many independent trials to halve that standard error. This reduces estimation error; it does not reduce the quantum uncertainty of a single prepared copy.

For the spin expectation value, let $n_{+}$ and $n_{-}$ be the two outcome counts, with $N=n_{+}+n_{-}$. ThenThe binomial distribution gives the exact sampling model. For $\theta =\pi /3$, the standard error is $\hslash \sqrt{3}/(4\sqrt{N})$: approximately $0.1083\hslash$ for $N=16$ and $0.03827\hslash$ for $N=128$. A run can fluctuate away from the predicted mean even as its typical spread decreases.

Repeatedly measuring one surviving system is a different protocol. Under the ideal Lüders rule, outcome $a_r$ leaves $P_r|\psi ⟩/\sqrt{p_r}$ when $p_r>0$. With no intervening change of observable or dynamics moving the state out of that eigenspace, the next projective measurement returns $a_r$ with certainty. Such a record is correlated and does not sample the original Born rule distribution independently. The quantum nondemolition measurement discussion describes this repeatability; fresh quantum state preparations supply the ensemble needed for the estimator above.

The expected value premium principle charges the expected value of the covered loss plus a proportional margin. For a nonnegative random variable $L$ with finite expected value, and a fixed dimensionless loading $\theta \ge 0$, it setsHere $\theta =0$ gives the expected claim cost alone; $\theta >0$ gives a positive margin when $\mathbb{E}[L]>0$. The loading is a fraction of the expected value, not of the largest possible claim size. Specify the covered period and use a consistent monetary valuation date. The rule itself does not determine $\theta$. ^[ref].

Consider an explicitly toy one-year policy: it pays £1,000 with probability $p=0.1$, and zero otherwise. Write $L=1000B$, where $B$ has the Bernoulli distribution with parameter $p$. The expected value is $1000(0.1)=\pounds 100$. A loading $\theta =0.2$ adds £20, making the premium £120. Neither the £100 expected value nor the £120 premium is a possible realized claim: the actual payment is £0 or £1,000. The first diagram separates these quantities. All numerical examples and both diagrams here are exact consequences of invented assumptions, not measured insurance data.

Why pay £120 for a loss whose expected value is only £100? Conditional on the insurer meeting its obligation, the policyholder replaces an uncertain £1,000 bill with a known £120 payment. The insurer can combine many such policies, so a few claims are funded by premiums from the whole pool. Winkel develops this motivation and the importance of independence in §15.2 of ^[ref].

To see the pooling mechanism precisely, suppose there are $n$ toy policies with mutually independent Bernoulli random variables $B_1,\dots ,B_n$, each of parameter $0.1$. Every claim size is £1,000, with at most one claim per policy. The aggregate claims model isUsing linearity of expectation and addition of variances for independent random variables, the expected value and standard deviation of this binomial distribution aggregate areThus 100 policies have expected aggregate claims £10,000 and aggregate standard deviation £3,000; 10,000 policies have expected aggregate claims £1,000,000 and aggregate standard deviation £30,000. Total uncertainty grows, but much more slowly than the expected value. Per policy, $S_n/n$ has expected value £100 and standard deviation $\pounds 300/\sqrt{n}$. Equivalently, the aggregate coefficient of variation, its standard deviation divided by its positive expected value, isIt falls from $300\%$ for one policy to $30\%$ for 100 policies and $3\%$ for 10,000. This is the useful beauty of independent pooling: an erratic individual cost becomes a comparatively stable average cost. The law of large numbers describes the corresponding convergence of the average toward its expected value.

At the same fixed loading $\theta =0.2$, total premium is $\Pi (S_n)=120n$, with expected surplus before other outgoings $120n-100n=20n$. The loading margin divided by the aggregate standard deviation is $20n/(300\sqrt{n})=\sqrt{n}/15$. This ratio rises with $n$ under our independence assumption; it is not a specified loss probability. For $n=100$, premiums total £12,000, but 13 claims cost £13,000. The one-year probability that claims exceed collected premiums is exactlyA positive loading therefore leaves about a $19.8\%$ chance of a one-year claims deficit in this toy pool. A claims deficit need not mean insolvency if capital is available. This calculation assumes premiums are collected before claims and ignores expenses and investment income; it is not an ultimate ruin probability.

The expected value premium principle also has a revealing limitation. With a common fixed $\theta$, linearity of expectation gives $\Pi (L_1+L_2)=\Pi (L_1)+\Pi (L_2)$ even without independence. Pooling therefore does not automatically reduce the premium per policy under this rule, and two losses with the same expected value receive the same premium regardless of their variance or large-loss probabilities. Any decision to change the loading after pooling requires information beyond the formula.

Dependence can remove the stability benefit. For arbitrary square-integrable losses, the aggregate variance includes their covariances:In the extreme toy common-shock model, every policy has $L_i=1000B$ for the same Bernoulli random variable $B$. Each policy still has expected value £100, so total premium is still $120n$, but $S_n=1000nB$ has standard deviation $300n$ and coefficient of variation $3$ for every $n$. With probability $0.1$, all policies claim together and losses exceed premiums by $880n$. The second diagram compares this model with independent pooling. Real shared events can invalidate independence; Winkel uses flood claims as an example in §15.2 of Actuarial Science II. The common-shock curve is an illustrative limiting model, not a fit to flood data.

The earlier $n^{-1/2}$ relative-variability calculation needs finite variance and the stated model of independent and identically distributed random variables. A finite expected value alone suffices to define the premium but does not imply finite variance. The rule makes no promise of solvency: losses, available capital and the timing of payments must be modelled separately. In the classical risk model, claim arrival rate $\lambda$ and mean claim size $\mu$ give premium rate $c=(1+\theta )\lambda \mu$; relative safety loading is then $\theta$. That model tracks capital through time and asks for ultimate ruin probability, a different question from calculating an expected margin. ^[ref]. Under quota share reinsurance, the reinsurer can apply the expected value premium principle to its ceded fraction $(1-\alpha )S$ using its own loading, which need not equal the direct insurer's loading.

A radius-$r$ Hamming decoder outputs the unique codeword within distance $r$ of the received word, if there is one. Ambiguous cases and absent candidates can be declared errors, or assigned arbitrarily for an error bound. The average error is the mean of the conditional error probabilities over the equally likely transmitted messages.

Let $V_n(r)={\sum }_{j=0}^r\left(\genfrac{}{}{0ex}{}{n}{j}\right)$ be the size of a Hamming ball. For any transmitted random codeword, the channel noise weight is $U\sim \mathrm{Bin}(n,p)$. Decoding can fail only if $U>r$ or another codeword lies within distance $r$ of the output. Conditional on the output, each independently sampled competing codeword is still uniform on the whole word space, so has probability $2^{-n}{V}_n(r)$ of lying in that ball. A union bound yieldsThis is an average over the random code construction as well as over messages and noise. It implies existence of at least one deterministic codebook with average error no greater than this bound, rather than asserting the same inequality for every randomly sampled book.

Choose a target rate $R<1-h(p)$ and a small $\delta >0$ such that $p+\delta <1/2$ and $R<1-h(p+\delta )$. Let $M_n=\lfloor 2^{nR}\rfloor$ and $r=\lfloor n(p+\delta )\rfloor$. The standard Hamming-volume estimate is $V_n(r)\le 2^{nh(r/n)}$: for $s=r/n\le 1/2$, each binomial probability term $s^j(1-s{)}^{n-j}$ with $j\le r$ is at least $s^r(1-s{)}^{n-r}$, and summing proves this bound. Thus the union-bound term decays exponentially, while $P(U>r)\to 0$ by the law of large numbers or a Chernoff bound.

There exists a $2M_n$-message book with average error ${\epsilon }_n\to 0$. At least half its messages have individual error at most $2{\epsilon }_n$, by Markov's inequality on the uniformly chosen message. Keep $M_n$ of these and discard the rest. Restricting the decoder's candidate set cannot spoil a previously unique successful decoding. The retained book therefore has maximum error at most $2{\epsilon }_n\to 0$. For large $n$ its retained codewords must be distinct: identical words could not both have conditional error below one half. This expurgation of a codebook proves reliable communication at every rate below $1-h(p)$, while converting an average-error estimate into the requested maximum-error conclusion.

For fixed prior parameters, divide numerator and denominator by $n$:under independent sampling from a geometric distribution with true success probability $p_0>0$. The law of large numbers gives $S/n\to 1/p_0$. In a large finite sample the posterior mean is close to $n/S=1/\bar{x}$, the maximum-likelihood estimate. The fixed prior's influence vanishes, and the estimate is consistent.

Under the null, replace both parameters by a common $r$. The likelihood becomesThe pooled maximum-likelihood estimate isUnder an interior null $p=q=r$, part (ii) gives a centered asymptotic normal difference with variance $r(1-r)(1/(2M)+1/(2N))$. The law of large numbers makes $\hat{r}\to r$ in probability. By Slutsky's theorem, replacing $r$ by this consistent estimate givesFor the two-sided alternative, reject when $|Z|>z_{1-\alpha /2}$, the corresponding standard-normal quantile. This has asymptotic significance level $\alpha$.

Put $A={\sum }_i{x}_i$, $B={\sum }_i{x}_i^2$ and $M_2=B/n$. The normal distribution model givesand, up to a constant independent of $\mu$, its log-likelihood function isThe score function and its zero equation areFor $B>0$, this quadratic equation has exactly one positive root, giving the maximum-likelihood estimatorThe numerator $B-A\mu -n{\mu }^2$ of the score function is positive between its negative and positive roots and negative above its positive root. Thus the likelihood function increases up to $\hat{\mu }$ and decreases afterwards: this is its unique global maximum on $(0,\infty )$. Also, $\ell (\mu )\to -\infty$ at both ends. The all-zero observation vector has probability zero; on that exceptional vector $L(\mu )\propto {\mu }^{-n}$ and there is no attained maximum.

The second derivative and the relevant moments areTherefore the expected Fisher information isThe support of a probability distribution is the whole real line, independent of the positive statistical parameter. The law of large numbers gives $\overline{X}\to \mu$ and $M_2\to 2{\mu }^2$ in probability, so the displayed maximum-likelihood estimator is a consistent estimator. The regular asymptotic normality of a maximum likelihood estimator now yieldsThus the requested leading asymptotic variance on the unscaled estimator's scale is ${\mu }^2/(3n)$. The normal mean proportional to standard deviation model provides a reusable version of this calculation.

The specified asymptotic relative efficiency isFor every fixed $\theta >0$ this lies strictly between zero and one. Prefer $T_2$ on grounds of asymptotic precision: it has the smaller leading asymptotic variance and attains the inverse Fisher information. Both are consistent estimators, by the law of large numbers and the continuous mapping theorem applied respectively to $\bar{X}$ and $A/n$. The efficiency approaches zero as $\theta \downarrow 0$ and one as $\theta \to \infty$.

This preference is an asymptotic comparison. In particular, it does not assert that $T_2$ is an unbiased estimator: since $A$ has a gamma distribution of shape $n$ and rate $\theta$, $\mathbb{E}{T}_2=n\theta /(n-1)$ for $n>1$.

Probability theory describes uncertainty through a probability space: possible outcomes form a sample space, observable questions are events, and a probability measure assigns them numbers between zero and one. The Kolmogorov axioms make those assignments consistent. They let us combine information about several events, update it through conditional probability, and study random variables without listing every possible outcome separately.

For example, roll two dice whose results are independent random variables, each with a discrete uniform distribution on $\{1,\dots ,6\}$. There are 36 equally likely ordered outcomes. Only $(1,6),(2,5),\dots ,(6,1)$ have sum seven, so that event has probability $6/36=1/6$. The sum itself is not uniformly distributed: sum two has only one outcome. Choosing a probability distribution for the elementary outcomes is part of the model; equal likelihood does not follow merely from counting the named possibilities.

The subject becomes especially useful when complete outcome lists are enormous or infinite. Independence turns a joint calculation into simpler pieces, expected value summarizes a random variable, and the law of large numbers explains how averages can become predictable even when individual observations are uncertain. The central limit theorem describes the scale and shape of many remaining fluctuations. These are distinct claims: a small variance of an average does not make every observation close to its expected value.

There is a satisfying unity here: dice, hidden causes, and Brownian motion all use the same rules for events and conditional probability. A model still needs justification. Its exact deductions describe what its assumptions imply; comparison with observations belongs to statistical inference.