Random variables converge in probability to when for every fixed . It makes a fixed-size error unlikely at a late deterministic time; it does not assert that every path eventually avoids such errors. For uniform on , converges to zero in probability since its probability of being one is . Almost sure convergence and convergence in Lp each imply convergence in probability. No shared rate is part of the definition, and convergence of expected values requires further conditions.
For real random variables on one probability space, fix a continuity point of the cumulative distribution function . For every , inclusions of events giveTake the lower and upper limits as , using convergence in probability, and then let . Continuity at makes both bounds , proving convergence in distribution. The two modes of convergence have different information: convergence in probability concerns the coupling on one probability space, whereas convergence in distribution concerns only the marginal probability distributions.
For , Markov's inequality gives the displayed bound, so implies convergence in probability to zero even when the first absolute moment does not tend to zero. For products of independent and identically distributed factors, it suffices that . With the uniform distribution on , the product of independent uniform random variables formula makes this . Since as , such an exists whenever .
Uniform convergence in probability means that the supremum of the absolute discrepancy on the entire index set tends to zero in probability. It controls evaluation at random indices in that set, unlike pointwise convergence in probability. On a fixed compact index set it is a special case of uniform convergence on compacts in probability.
A sequence of random variables is bounded in Probability when . A sequence converging in probability to a finite random variable has this property, and its product with a sequence tending to zero in probability also tends to zero in probability.
The notation means that is bounded in probability asymptotically: for every some gives . A tail bound is sufficient. In statements involving bandwidths, specify whether the constants must be uniform over bandwidth sequences.
A bound on does not give the same bound on . Even independent standard normal random variables each have uniformly bounded tails while their Gaussian maximum over indices grows at scale . This distinction matters in nonparametric statistics when a shrinking bandwidth permits many spatial windows.
If for fixed real , convergence of the distribution functions at and shows . Thus convergence in distribution to a deterministic limit implies convergence in probability to that limit, even though the implication fails for nonconstant limits on a specified probability space.
Let be real independent and identically distributed random variables with and expected value . The weak law of large numbers states that their sample mean has convergence in probability to : for every fixed ,Thus a large sample is unlikely to give an average with a specified fixed-size error. The strong law of large numbers makes the stronger statement of almost sure convergence along an entire sequence. Under the displayed assumptions both laws apply; their names distinguish the conclusions, not two competing predictions.
When the variance is finite, variance additivity for independent random variables gives . The Chebyshev inequality immediately supplies a useful quantitative form and a brief proof:For independent trials with a Bernoulli distribution of parameter , the expected value is and the variance is . At , the chance of an error of at least is therefore at most . The exact binomial distribution gives a much smaller probability, as the figure under the strong law of large numbers shows. The bound is conservative, but it is valid for every sample size under these assumptions and does not require a normal approximation.
The weak law of large numbers addresses a fixed tolerance . The central limit theorem instead resolves errors on the shrinking scale and gives their approximate normal distribution. In particular, the weak law of large numbers alone supplies neither a bell-shaped error law nor a universal rate of approach. For example, a Pareto distribution with minimum one and shape has expected value three but infinite variance: both the weak law of large numbers and strong law of large numbers apply, while the finite-variance statement of the central limit theorem does not.
Suppose the variances of a sequence of random variables are bounded by and whenever , for a fixed nonnegative integer . The Cauchy-Schwarz inequality bounds each remaining covariance in absolute value by . In the variance of a sum, there are at most off-diagonal pairs, proving the displayed bound. The Chebyshev inequality makes the average minus its mean converge to zero in Probability. If all means equal , the average therefore converges in Probability to . independence of every summand is unnecessary.
Let have a symmetric probability distribution with for , mass at zero, and no mass with . Then , but independent copies satisfy the weak law of large numbers with limit zero. Indeed . Their symmetric truncations have mean zero and second moment , so Chebyshev's inequality gives in Probability. A union bound shows that the original and truncated sums differ with Probability tending to zero. This also gives a differentiable characteristic function at zero whose derivative cannot be interpreted as an absolutely convergent expectation.
For independent and identically distributed random variables with characteristic function , suppose their sample mean converges in Probability to a finite constant . Bounded exponential tests give uniformly on bounded intervals: the error is at most on . Taking the continuous complex logarithm vanishing at zero shows uniformly; an integer-valued branch difference is continuous and therefore zero. For arbitrary , put and , so . Dividing the uniform limit by proves , and thus . No absolute first moment is assumed.
Let be independent and identically distributed with probability law , and let be a class of measurable -integrable functions. Write and . A uniform law of large numbers concernswith measurability of the supremum assumed. A uniform weak law uses convergence in probability; a uniform strong law uses almost sure convergence. The mode and sampling assumptions are part of the statement.
For a finite class, the strong law of large numbers for each function holds simultaneously and gives the uniform strong law. For infinite classes, pointwise convergence alone is insufficient. For example, let be the uniform distribution on and take indicators of all finite subsets. Each fixed function has expectation zero and empirical mean tending to zero almost surely. Yet for every sample the finite set belongs to the class and its empirical mean is one. The displayed supremum is therefore one for every .
Conditions such as finite bracketing of a function class at every positive width control this complexity. The Glivenko-Cantelli theorem gives the uniform strong law for indicators of half-lines, the class determining the empirical distribution function.
A measurable-function class is pointwise separable if it has a countable subclass such that every member is the pointwise limit of a sequence from that subclass. With an integrable common envelope, dominated convergence theorem makes both empirical averages and population expectations converge along each such sequence. Thus the supremum of the absolute empirical errors equals the supremum over the countable subclass and is measurable. A finite integrable bracketing cover provides an integrable envelope by taking the maximum absolute value of all its endpoints.
A bracket contains every measurable function satisfying pointwise. For a Probability measure , its width is , with integrable endpoints. Finite bracketing at every positive width is a sufficient complexity condition for a uniform law of large numbers. Pointwise inequalities, or inequalities outside a single common null set, allow empirical inequalities to hold simultaneously throughout the class.
A measurable function is an integrable envelope of a function class for a class under a probability law if for every and . Such an envelope permits the dominated convergence theorem to control shrinking function brackets.
Suppose an integrable measurable-function class can be covered by finitely many function brackets of every positive width. For an independent sample with common law , the empirical measure satisfies on a common probability-one event. For one finite -cover, the supremum is bounded by plus the largest empirical error among its endpoints. The strong law of large numbers makes that finite maximum vanish. Taking a countable sequence of widths decreasing to zero proves the assertion. A measurable supremum can be obtained from a pointwise separable function class; otherwise the probability-one-event formulation expresses the same pathwise conclusion.
Let be a compact metric space, let be continuous in for each and measurable in for each , and assume the envelope is integrable under . For an independent and identically distributed sample,Compact metric structure supplies both finite parameter nets and countable dense sets, so the relevant suprema can be treated measurably under these continuity assumptions.
For a proof, define . Uniform continuity on the compact parameter set gives for each , and . Thus by the dominated convergence theorem. At a finite -net of centers, apply the strong law of large numbers to every center and to . For a nearby center ,The limiting supremum is at most . A countable sequence proves the claim. Equivalently, these center functions and their oscillation envelopes give the function brackets used in the uniform strong law from finite L1 bracketing.
For independent and identically distributed real observations with cumulative distribution function , the empirical distribution functionsatisfies the Glivenko-Cantelli theoremThe population law may have atoms; continuity of is not required. The conclusion is uniform over every threshold, strengthening the separate strong laws for fixed thresholds.[ref]
Here is a proof using inverse transform sampling. First take iid uniforms and their empirical CDF . At the finitely many grid points , the strong law of large numbers gives simultaneously. Monotonicity bounds the discrepancy between grid points by the largest grid discrepancy plus . Taking countably many proves uniform convergence on .
For general , construct using the quantile function. Then exactly when , so and the general supremum is bounded by the uniform one. This also handles jumps of . The theorem does not apply merely because an empirical CDF was formed: for repeated copies of the same nondegenerate random observation, the empirical CDF does not approach its population law in this way.
For a cumulative distribution function , let , . Right continuity gives exactly when , including at atoms. Thus applying to independent uniform variables constructs an independent sample with distribution function . Its empirical distribution function is , where is the uniform-sample empirical distribution function. Consequently . For the uniform sample, monotonicity and a grid of mesh bound the last supremum by the largest error at grid points plus . The strong law of large numbers on the countable union of all such grids proves the almost-sure limit. This construction proves the theorem for distributions with atoms as well as continuous distributions.
If converges in Probability to both and , then almost surely. Indeed, the triangle inequality and a union bound show that for every .
Convergence in probability is equivalent toThe forward implication splits at a fixed error threshold; the reverse implication is Markov's inequality.
independent indicators with success probabilities converge to zero in Probability, but successes occur infinitely often almost surely by the second Borel--Cantelli lemma.
Articles by others on the same topic
There are currently no matching articles.