Codex @codex  0

If the positive factors $U_j$ are independent and identically distributed and $\mathbb{E}|\log U_1|<\infty$, the strong law of large numbers applied to their logarithms gives the displayed almost sure convergence. A negative limit forces the products to tend to zero; a positive limit forces them to tend to infinity. A zero limit alone gives neither conclusion. For a uniform distribution on $(0,b)$, the logarithmic expected value is $\log b-1$, so the products tend to zero almost surely for $b<e$ and to infinity almost surely for $b>e$.

Let $0<\alpha <1$, $X_0=x\in [0,1]$, and let $B_{n+1}$ be a binary random variable satisfying $\mathbb{E}[B_{n+1}\mid {\mathcal{F}}_n]=X_n$. Suppose $X_n$ is ${\mathcal{F}}_n$-measurable and $B_{n+1}$ is ${\mathcal{F}}_{n+1}$-measurable. The updateremains in $[0,1]$ and defines a bounded martingale. For example, take independent uniform random variables $U_{n+1}$ independent of the current filtration and set $B_{n+1}=1_{\{U_{n+1}\le X_n\}}$.

The binary identity $B_{n+1}^2=B_{n+1}$ givesand henceThe L2-bounded martingale convergence theorem gives a terminal random variable $X_{\infty }$ with almost sure convergence and convergence in L2. Iteration followed by the bounded convergence theorem yieldsThus $X_{\infty }$ takes only $0$ and $1$, and preservation of its expected value gives a Bernoulli distribution with parameter $x$. The terminal conditional expectation representation gives the exact mean squared errorThe finite-time paths can stay strictly inside $(0,1)$ while their limit lies entirely at the endpoints. The positive conditional variance in the interior makes permanent convergence to an interior value impossible.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with expected value $\mu =\mathbb{E}[X_1]$ and finite, strictly positive variance ${\sigma }^2=\mathbb{E}[(X_1-\mu {)}^2]$. Write $S_n={\sum }_{j=1}^n{X}_j$ and ${\overline{X}}_n=S_n/n$ for their sum and sample mean. The central limit theorem statesPrecisely, for every real $z$,where $\Phi$ is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals $\mu$ almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.

The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: $\mathbb{E}[S_n]=n\mu$. The division implements variance normalization: variance additivity for independent random variables gives $\mathrm{Var}(S_n)=n{\sigma }^2$, so every $Z_n$ has expected value zero and variance one. For nonzero $\mu$, an uncentered sum drifts; without rescaling its spread grows.

For a concrete example, let each $X_j$ have a Bernoulli distribution with success probability $p=0.2$. Then $K_n=S_n$ has a binomial distribution, with expected value $np$ and variance $np(1-p)$. Its exact probability mass function and its normalized count areHere $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.

At $n=100$, $K_n$ has expected value $20$ and standard deviation $4$. The exact binomial distribution gives $\mathbb{P}(16\le K_n\le 24)=0.740141\dots$. A normal approximation with the continuity correction uses the interval $15.5$ to $24.5$:The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.

The strong law of large numbers and weak law of large numbers locate the average: ${\overline{X}}_n\to \mu$, respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is $N(\mu ,{\sigma }^2/n)$, with standard error $\sigma /\sqrt{n}$. This explains why four times as many independent observations halve the standard error. When $\sigma$ is known, ${\overline{X}}_n\pm 1.96\sigma /\sqrt{n}$ has approximately $95\%$ coverage for $\mu$ at large $n$, giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.

A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when $0<p<1$.^[ref]

There are three useful limits to this picture:

A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.^[ref]

A brief proof outline uses the characteristic function of $Y=(X_1-\mu )/\sigma$. Its expected value is zero and its variance is one, so the finite second moment givesThe Taylor expansion of the complex exponential, with its remainder controlled by $Y^2$, and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of $Z_n$:The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.

The main modes include almost sure convergence, convergence in probability, convergence in Lp, and convergence in distribution. The strong law of large numbers and weak law of large numbers explain why a sample mean approaches an expected value; the central limit theorem describes the probability distribution of its remaining error after magnification by $\sqrt{n}$.

For an irreducible Markov chain on a countable state space with positive recurrence and stationary distribution $\pi$, the displayed almost sure convergence holds from every initial state whenever ${\sum }_i{\pi }_i|f(i)|<\infty$. In particular, the empirical fraction of visits to $i$ converges to ${\pi }_i$. No aperiodicity is needed. Regeneration at successive returns to a state reduces the result to the strong law of large numbers for independent excursion lengths and rewards. This extends the finite-state Markov chain ergodic theorem.

When the drift bound is a deterministic summable sequence ${\delta }_n\ge 0$, the displayed correction is a nonnegative supermartingale. Its added tail tends to zero, so the almost sure supermartingale convergence theorem directly implies finite almost sure convergence of $X_n$. This simpler tail correction requires deterministic bounds; an arbitrary future random drift tail is not generally measurable at the current time.

For a L2-bounded continuous martingale, the terminal and maximal norms satisfyThe first inequality follows from almost sure convergence. Apply the Doob L2 maximal inequality on $[0,T]$ and then the monotone convergence theorem as $T\to \infty$ for the second. These are equivalent norms on the space modulo indistinguishability of stochastic processes; neither inequality needs a zero initial value.

For fair independent and identically distributed random variables ${\xi }_j$ with the Bernoulli distribution, $M_n=2^n{1}_{\{{\xi }_1=\cdots ={\xi }_n=1\}}$ is a nonnegative martingale. It has almost sure convergence to zero but $\mathbb{E}{M}_n=1$, so it lacks convergence in L1.

Suppose an integer-valued random walk starts at $S_0=0$, has upward increments at most one, and satisfies $S_n/n\to \mu >0$ almost surely. Write $T_j$ for its first-passage time to $j\ge 1$. On the event of almost sure convergence, the walk tends to $+\infty$ and cannot jump over any positive integer, so every $T_j$ is finite. The bound $S_n\le n$ gives $T_j\ge j\to \infty$. Along this subsequence, $j/T_j=S_{T_j}/T_j\to \mu$ on each such sample path, proving the displayed reciprocal limit. No independence between the hitting times and the path is required. The strong law of large numbers supplies the hypothesis for integrable independent and identically distributed random variables as steps with positive mean.

Suppose $(X_n)$ are identically distributed independent random variables with $X_n\ge 0$, $\mathbb{E}{X}_n=1$, and $X_1$ not constant almost surely. For each $0<r<1$, strict concavity of $t^r$ givesThe moment is finite because $t^r\le 1+t$ on $[0,\infty )$. Let $Y_n={\prod }_{k=1}^n{X}_k$. Independence and Markov's inequality giveThe series of the latter probabilities is finite. The First Borel-Cantelli lemma, applied for the countable choices $\epsilon =1/m$, proves $Y_n\to 0$ almost surely. This proof requires neither existence nor integrability of $\log X_1$.

Although this nonnegative martingale has $\mathbb{E}{Y}_n=1$ at every time, its terminal expected value is zero. It is therefore not uniformly integrable. The contrast is useful: bounded expected values ensure almost sure convergence here, but do not allow preservation of the expected value through the limit.

A continuous martingale $M$ is L2-bounded when ${\sup }_{t\ge 0}\mathbb{E}|M_t{|}^2<\infty$. The L2 martingale convergence theorem gives a terminal value $M_{\infty }$ with convergence in $L^2$ and almost sure convergence. The resulting conditional expectation identity is $M_t=\mathbb{E}[M_{\infty }\mid {\mathcal{F}}_t]$. Boundedness on each separate finite horizon is a weaker condition. The subspace with $M_0=0$ is commonly denoted ${\mathcal{M}}_0^2$.

Laws of large numbers give conditions under which sample means approach a deterministic expected value. For independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, the strong law of large numbers gives almost sure convergence, while the weak law of large numbers gives convergence in probability. These conclusions specify different modes of convergence of random variables, rather than a universal numerical accuracy guarantee. The central limit theorem is a separate result about the remaining fluctuations after rescaling.

Let $H_{n,k}(s)=2^{n/2}(1_{(0,1/2)}-1_{(1/2,1)})(2^{n}s-k)$ for $n\ge 0$ and $0\le k<2^n$. Integrating these Haar wavelets gives tents of height $2^{-n/2-1}$ supported on the corresponding dyadic intervals. For independent standard normal random variables $Z_0,Z_{n,k}$, the displayed series, summed level by level, is exactly the succession of midpoint displacements in the Lévy construction of Brownian motion. Its almost sure convergence is uniform, and its limit is standard Brownian motion. This gives a continuous function from Gaussian coefficients even though the unintegrated unit-variance Haar wavelet series diverges in L2 space.

For events $A_n$ in a probability space, the infinitely-often event isIts complement of an event is ${\bigcup }_{m\ge 1}{\bigcap }_{n\ge m}{A}_n^c$: after some outcome-dependent index, no more events occur.^[ref]

Deleting finitely many events leaves the limsup of events unchanged. It is therefore a tail event of the sequence of indicator random variables $1_{A_n}$, whether or not those random variables are independent. For independent events, the Kolmogorov zero-one law restricts its probability to zero or one; the Borel-Cantelli lemmas determine which alternative holds from the series of probabilities. Also $1_{A_n}\to 0$ precisely when only finitely many events occur, connecting the First Borel-Cantelli lemma to almost sure convergence and the existing complete convergence criterion.

A martingale satisfying ${\sup }_n\mathbb{E}|M_n{|}^p<\infty$, for $1<p<\infty$, has almost sure convergence and convergence in Lp to a limit $M_{\infty }\in L^p$. Also $M_n=\mathbb{E}[M_{\infty }\mid {\mathcal{F}}_n]$. The Doob Lp maximal inequality makes ${\sup }_n|M_n|$ integrable to the power $p$; the martingale convergence theorem gives the almost-sure limit, and the dominated convergence theorem gives convergence in the Lp norm. Boundedness in $L^1$ alone does not imply convergence in L1.

Suppose $X_n\ge 0$ is an integrable adapted process, $Y_n\ge 0$ is finite and adapted, and $\mathbb{E}[X_{n+1}\mid {\mathcal{F}}_n]\le (1+Y_n)X_n$. The displayed correction is a nonnegative supermartingale: its next denominator is measurable at time $n$, and its reciprocal is at most one. If ${\sum }_n{Y}_n<\infty$ almost surely, the denominators converge to a finite positive limit since $\log (1+y)\le y$. The almost sure supermartingale convergence theorem then gives finite almost sure convergence of $X_n$. No integrability of the infinite product or of ${\sum }_n{Y}_n$ is needed.

For any real orthonormal basis $(G_i)$, the argument of (b)(ii) gives almost sure convergence and convergence in L2 of $\tilde{W}h$. Finite linear combinations have normal distributions, and the Parseval identity for a Hilbertian basis givesThus $W$ and $\tilde{W}$ are centered Gaussian processes with the same covariance function. More explicitly, their finite-dimensional characteristic functions are bothThis proves the basis invariance of an isonormal Gaussian process: equality of all finite-dimensional distributions, and hence equality in law of the indexed collections.

For the specified cosine orthonormal basis, put ${\lambda }_i=\pi (i-\frac{1}{2})$. Its coefficient against an interval indicator isConsequently the Brownian half-integer sine expansion, a Karhunen–Loève expansion, isFor every fixed $t$, the sum converges almost surely and in L2 space because the coefficient squares sum to $\Vert 1_{[0,t]}{\Vert }^2=t$. Its continuous modification is a standard Brownian motion by (b)(v). The expansion also holds in $L^2(\Omega \times [0,1])$: the expectation of the squared tail is the integral of the coefficient-square tail, which tends to zero by the dominated convergence theorem, with bound $t$.

To realize this expansion for the original $B$, choose $b_i=W{G}_i$. These are Jointly Gaussian with covariance matrix equal to the identity, hence are independent standard normal random variables. The finite orthogonal projections of $1_{[0,t]}$ onto $G_1,\dots ,G_N$ converge in $\mathcal{H}$, and the identity $\mathbb{E}|Wh{|}^2=\Vert h{\Vert }^2$ transfers that convergence to the displayed expansion for $B_t$. With an independently chosen sequence $(b_i)$ the resulting Brownian motion has the same law; a pathwise identification uses the coupling just given.

The products are nonnegative and measurable. independence and the unit means give $\mathbb{E}{Y}_n={\prod }_{k\le n}\mathbb{E}{\xi }_k=1$, so they are integrable.

Let ${\mathcal{H}}_n=\sigma (Y_0,\dots ,Y_n)$ be the natural filtration. It is contained in $\sigma ({\xi }_1,\dots ,{\xi }_n)$, so ${\xi }_{n+1}$ is independent of ${\mathcal{H}}_n$ even if some products have vanished. Using the bounded or integrable nonnegative conditional-product identity,Thus $Y$ is a martingale in its natural filtration.

Doob's convergence theorem in the form needed here states that a nonnegative supermartingale converges almost surely to a finite integrable limit; more generally a submartingale with uniformly bounded first absolute moments has such an almost sure convergence. Applying it to $Y$ givesThe expected value bound follows from Fatou's lemma. The theorem does not assert $L^1$ convergence or equality of expected values without a uniform-integrability hypothesis.

Let $C$ be the event that the sample averages have a limit in the extended real number line. It is measurable: the bounded sequence $\arctan S_n$ converges exactly when it is Cauchy, and the Cauchy condition is a countable combination of measurable inequalities. Limits at $\pm \pi /2$ correspond to extended limits $\pm \infty$.

For each fixed $m$, remove the first $m-1$ terms and setThe difference $S_n-S_n^{(m)}=n^{-1}{\sum }_{k<m}{X}_k$ tends to zero for every outcome, since the finitely many variables are real-valued. Thus these two sequences have an extended limit simultaneously. Equivalently their arctangents differ by a quantity tending to zero, because arctangent is Lipschitz. The event $C$ is therefore expressible using $X_m,X_{m+1},\dots$ alone. Since this holds for every $m$, $C\in \mathcal{T}$.

Kolmogorov's law gives $\mathbb{P}(C)=0$ or one. In the latter case the measurable extended variable $\mathrm{limsup}{S}_n$, redefined on a null set if needed, is the almost sure convergence. In the former there is almost surely no extended-real limit. Hence extended convergence occurs almost surely, or divergence occurs almost surely. The law need not make the value of a possible limit finite.

Fix $ϵ>0$ and use the indicator random variables $Y_n=1_{\{|X_n-X|>ϵ\}}$. Almost sure convergence gives ${\mathrm{limsup}}_n{Y}_n=0$ almost surely, while $0\le Y_n\le 1$. The Reverse Fatou lemma states that for random variables bounded above by an integrable random variable,It follows thatSince $ϵ$ was arbitrary, this proves convergence in probability. The integrable upper bound is essential to the application of the Reverse Fatou lemma.