Almost-sure convergence and convergence of second moments Created 2026-10-03 Updated 2026-11-05
If square-integrable random variables satisfy almost surely, then in exactly when . Boundedness in and almost-sure convergence first give weak convergence in the Hilbert space ; weak convergence together with convergence of norms gives strong convergence.
David Pollard 2026-11-05
David Pollard is a statistician whose research and exposition concern weak convergence, empirical processes and stochastic processes. Yale's institutional profile identifies these specialties, together with statistical asymptotics and econometrics.[ref]
A recurring scientific issue in his writing is how bounds for one random quantity become simultaneous bounds for a whole indexed family. His Measurability Woes by David Pollard explains why pointwise almost-sure agreement does not automatically provide a single exceptional set valid at every index. That distinction matters when a supremum or an entire sample path enters an argument. His author page lists Convergence of Stochastic Processes (1984) and Empirical Processes: Theory and Applications (1990), followed by broader notes on probability methods.[ref][ref]
Use the direct method in the calculus of variations. The affine function is admissible with finite energy, and the energy is nonnegative since is even. Thus is finite. Choose a minimizing sequence with uniformly bounded energy. Then
The fixed value and the Cauchy-Schwarz inequality give and . Hence the sequence is bounded in and uniformly bounded and equicontinuous.
By reflexivity and the Arzelà-Ascoli theorem, pass to a subsequence with in and uniformly. The two limits coincide; uniform convergence also preserves , . Since all functions stay in a common bounded interval, uniform convergence gives uniformly, and therefore convergence of the potential integrals. The derivative term is lower semicontinuous for weak convergence, so
Admissibility gives . Thus
We prove existence by the direct method in the calculus of variations, making the step with weakly compact sets explicit. For and , introduce the functional
It is finite by part (b)(i). Hölder's inequality and the Sobolev embedding theorem give
Thus is a coercive function, bounded below, and every minimizing sequence is bounded in .
The space is a reflexive Banach space: it is isomorphic to a closed linear subspace of the finite product of the reflexive spaces recording the function and its weak derivatives. A bounded sequence in a reflexive Banach space has a weakly convergent subsequence. Hence, after passing to a subsequence, a minimizing sequence satisfies in . The continuous linear maps into and into preserve weak convergence, so
The norm in any Banach space has weak lower semicontinuity, since it is the supremum of absolute values of bounded linear functionals of dual norm at most one, by the Hahn-Banach theorem. Therefore the two nonnegative norm powers in have weak lower semicontinuity. Also belongs to and is continuous for weak convergence. It follows that
For any , the real function has a minimum at . Its derivative there is
This differentiation is justified directly by finite polynomial expansions: the fourth power of the Euclidean norm of and the eighth power of have integrable coefficients by Hölder's inequality. Thus , giving existence of a weak solution.
If are two weak solutions, testing the difference by and using part (b)(iii) gives
Hence
No sign assumption on or on the weak solution is required.
The weak topology is the coarsest topology making every continuous. Its basic neighbourhoods impose finitely many inequalities . The weak-star topology is the coarsest topology making each evaluation continuous, for .
The canonical embedding into the bidual is
It is linear and . For , the Hahn-Banach theorem supplies a norming functional with and , proving equality. Thus is an isometric embedding. The coordinates defining the weak-star topology on , restricted to , are precisely for . These are exactly the coordinates defining the weak topology on , so is also a topological embedding for these topologies.
A normed vector space is a reflexive space if is onto . Such a space is automatically a Banach space, since it is isometric to the complete continuous dual space . The Banach-Alaoglu theorem states that the closed unit ball of a continuous dual space is compact for its weak-star topology. The Goldstine theorem states that is dense in the weak-star topology in . Consequently
If is onto, its weak-to-weak-star homeomorphism carries onto the Banach-Alaoglu theorem's compact ball . Conversely, if is weakly compact, is compact and hence closed in the Hausdorff weak-star topology. Its density from the Goldstine theorem makes it all of , and scaling gives surjectivity of .
Now suppose is separable and nonzero. Choose a countable dense set in its unit sphere and, using the Hahn-Banach theorem, choose norming functionals with and . These continuous linear functionals separate points: for , choose to get . Define
The series converges uniformly, separates distinct points, and obeys the triangle inequality because . Thus is a metric. Its topology is coarser than the weak topology: a finite set of coordinate controls makes the finite initial sum small, while the remaining sum is at most . On a weakly compact set , the identity is therefore a continuous bijection from a compact space into a Hausdorff space, hence a homeomorphism. This proves that is metrizable in its weak topology, an instance of countable separating family metrizes a weakly compact set.
If is also a reflexive space, its closed unit ball is weakly compact and therefore metrizable by this construction. Compactness in a metric space implies sequential compactness, so every sequence in has a weakly convergent subsequence. Scaling proves the claim for every bounded sequence.
For the converse, assume every bounded sequence in has a weakly convergent subsequence. Use the same metric on . A weak limit of points of remains in : for every , and the Hahn-Banach theorem recovers . Thus every sequence in has a subsequence convergent in to a point of , so is sequentially compact and hence compact.
We must still recover the entire weak topology, rather than just the selected coordinates. Suppose in but not weakly. There is a neighbourhood in the weak topology of and a subsequence outside . By hypothesis that subsequence has a further subsequence converging weakly to . It then converges in to , so uniqueness of metric limits gives , contradicting eventual membership in . Hence convergence in implies weak convergence. Every is therefore sequentially continuous on and hence continuous, since the domain is a metric space. The weak topology and -topology on coincide. The ball is weakly compact, so the criterion above proves
The zero space satisfies all statements directly. The converse did not assume completeness, nor use the full Eberlein-Šmulian theorem.
The weak topology is coarser than the norm topology, so norm convergence implies weak convergence, and
For the reverse inclusion, let . The set is closed and convex. The Hahn-Banach separation theorem supplies and such that . The set is weakly open, contains and misses . Hence is outside the weak closure as well. If is empty the equality is immediate. Therefore
The separating functional explains the role of convexity: an arbitrary norm-closed subset need not be weakly closed.
By weakly convergent sequence is bounded, is finite. Choose any . Once have been chosen, put . Since is fixed, weak convergence to zero gives . Choose with . This gives the required one-sided inequality as well.
The useful estimate is for the squared norm. Indeed, , and the inner product expansion gives
By mathematical induction, . Hence
This proves norm convergence of the averages, the weak Banach–Saks theorem in a Hilbert space. The printed inequality omits the square on its left side. Our stronger estimate also proves that printed bound, since for , but the printed bound alone only bounds the averages and cannot imply their norm convergence to zero.
Define the continuous linear functionals . For each fixed , weak convergence makes a convergent, hence bounded, scalar sequence. The Uniform boundedness principle states that a pointwise bounded family of bounded linear operators from a Banach space to a normed space has uniformly bounded operator norms. Since a Hilbert space is a Banach space, it applies to .
The Cauchy-Schwarz inequality gives , and evaluation at gives equality when ; equality also holds at zero. Consequently
First suppose the convex set is closed in the norm topology, and satisfies . Apply the weak Banach–Saks theorem in a Hilbert space to the weakly null sequence . Along the resulting subsequence,
in norm. Each is a convex combination of points in , so . Since is a closed set in the norm topology, .
Conversely, suppose is a sequentially weakly closed set. If in the norm topology, choose with . The Cauchy-Schwarz inequality gives for every , hence weak convergence to . The assumed closure property gives . The empty set satisfies both properties too. Thus
For each real , define the sublevel set . Continuity of and closure of make a closed set in ; the convex function inequality makes it a convex set. The preceding result therefore makes every a sequentially weakly closed set. Equivalently, has weak lower semicontinuity along sequences in .
Every is bounded. This is immediate if is bounded. Otherwise, an unbounded would contain with , contradicting the assumed coercive function condition .
Choose and put . Choose a minimizing sequence entirely in , with ; when , require for sufficiently large . It is bounded by the previous paragraph.
The weak subsequence of a bounded Hilbert-space sequence theorem gives . This applies even if is not separable: the closed linear span of the sequence is a separable Hilbert space; successive extraction makes every coefficient in a countable Hilbertian basis converge, the common norm bound gives square-summable limiting coefficients by the Bessel inequality, and approximation by finite linear combinations proves weak convergence. Orthogonal decomposition extends that conclusion to all test vectors in . Since is a sequentially weakly closed set, .
For any real , a tail of lies in , so . If , this would give for every real , impossible because is real-valued. Thus is finite; letting gives , while the definition of infimum gives the reverse inequality. Therefore
This proves minimum of a coercive convex functional on a Hilbert space by the direct method in the calculus of variations, with bounded sublevel sets supplying the required weak sequential compactness in a Hilbert space.
Take the weakly convergent subsequence in (i), with limit . For each , the weak formulation gives
The left side tends to by weak convergence in . The zero-boundary Sobolev inequality gives , so (ii) makes the right side tend to zero. Taking in the limiting identity yields . The zero-boundary Poincaré inequality then gives .
The same argument applies to a weakly convergent subsequence of any subsequence of , because all remain bounded as in (i). To see why this proves convergence of the full sequence, suppose a continuous linear functional on did not satisfy . Some subsequence would have . Its boundedness gives a further weakly convergent subsequence, whose limit has just been proved to be zero; then tends to zero along it, a contradiction. Therefore
Uniqueness of a possible limit alone is not enough without the weak sequential compactness in a Hilbert space used to extract a further subsequence.
The Lp norm of the forcing at the critical exponent is constant:
To prove weak convergence in , the duality of Lp spaces says that we must test against an arbitrary . Holder inequality, restricted to the shrinking measurable set , gives
The last step is absolute continuity of the Lebesgue integral, since and is integrable. Thus
Testing only bounded or smooth functions would not by itself establish this weak convergence; the restricted Holder inequality handles every member of the full dual space.
Let . By linearity of the weak formulation, is the zero-boundary weak solution with forcing . The estimate in (a)(iii) therefore gives
The change of variables gives , so the division is legitimate. Since is a Hilbert space, weak sequential compactness in a Hilbert space now gives a subsequence and some such that
In fact : the Sobolev trace operator is continuous and linear, so its zero kernel is preserved by weak convergence.
Let be the convex hull of all terms of the sequence, meaning the set of their finite convex combinations. Suppose . Part (ii) then gives a continuous linear functional with for every . In particular for every , contradicting . Therefore the ball meets .
A point of that intersection is a finite combination with nonnegative coefficients summing to one. Set , combine repeated indices and insert zero coefficients for omitted indices. This gives
Thus zero lies in the norm closure of the convex hull. The same argument applies to every tail of the sequence and, by choosing errors tending to zero, also gives the usual Mazur lemma. Completeness of is not required for this conclusion.
The hypothesis needed is precisely convergence to zero against every continuous linear functional, namely weak convergence. The PDF says only “continuous” for the testing maps. If that is interpreted as including arbitrary nonlinear continuous maps, the constant map makes the hypothesis impossible. The proof above establishes the substantive conclusion under the intended, weaker linear-testing hypothesis, and therefore also the implication under the literal stronger wording.
If and , then because is bounded and linear. Therefore , proving weak continuity of .
For a bounded sequence in , extract successive subsequences on which the first, second, and subsequent coordinates converge, then take a diagonal subsequence. Call the coordinate limits . Every finite partial sum satisfies , so and . For any , the pairing on its first coordinates converges, while the pairing on the tail is bounded uniformly by . First choose large, then take the subsequence limit. The Riesz representation theorem identifies these pairings with every continuous linear functional, proving the requested weak convergence.
For , norm convergence implies weak convergence immediately. Conversely, suppose but a subsequence has . Coordinate convergence to zero allows us to select indices and cutoffs such that the mass of before or at is below , and its mass after is below . Its mass on the intervening block is then at least . On that block set when nonzero and zero otherwise. These prescriptions define . The bounded linear functional has real part at least on every selected , contradicting weak convergence. Translating by the limit proves weak and norm convergence of sequences coincide in , the Schur property.
A compact operator sends bounded sets to relatively norm-compact sets. Every bounded sequence in has a weakly convergent subsequence by the argument above, after scaling into the unit ball. Its image under converges weakly and therefore in norm by the Schur property. Sequential compactness in a metric space gives relative compactness of the image of the unit ball. Every bounded linear operator from to is compact.
The required functional, with the sign appropriate to a positive divergence operator, is
It is differentiable on , with . A critical point consequently satisfies the weak equation. We now prove existence by the direct method in the calculus of variations, rather than invoke the representation theorem again.
The ellipticity and Poincare estimates imply
Thus the infimum is finite and any minimizing sequence is bounded in the Sobolev norm. A bounded sequence in a Hilbert space has a weakly convergent subsequence; write in . The equivalent norm has the same continuous linear functionals and hence the same weak convergence. A Hilbert norm is Weak lower semicontinuity, giving . The term converges to because it is a bounded linear functional. Therefore
The limit attains the infimum. Differentiating at its minimum, for every , gives . This produces the desired weak solution. Strict positivity of also makes the functional strictly convex, consistent with uniqueness.
Suppose in . Then in . Since is a differentiable convex function, its supporting inequality is
Take , and integrate. Part (i) shows that every energy is finite and that the fixed field belongs to , so
The last integral tends to zero by the definition of weak convergence in . Therefore
This is weak lower semicontinuity of convex gradient energies. It uses a fixed supporting gradient at the limit; weak convergence alone would not justify passing pointwise through the nonlinear integrand.
Assume the bounded set is measurable, as needed to define its indicator in . Put and choose a smooth cutoff function of compact support equal to one near . Weak convergence in the H1 space gives a uniform bound, by the Uniform boundedness principle, and therefore is uniformly bounded in with fixed compact support.
Here is a direct Fourier proof of the local compactness in the Rellich-Kondrachov compactness theorem. For each frequency , , since integrating against the fixed compactly supported function is a continuous linear functional on . Also uniformly, by Cauchy-Schwarz inequality on the fixed support. The dominated convergence theorem gives, for every fixed ,
The uniform derivative bound and the unitary Plancherel theorem control high frequencies:
First let , then . This proves , and hence .
Let . The Sobolev embedding theorem bounds uniformly. For , choose with . Interpolation of Lp norms gives . For , Holder inequality on the fixed finite-measure support gives the same conclusion from convergence. Thus
The same integral estimate covers if those spaces are interpreted with their usual quasi-norm. No conclusion at the critical exponent is asserted.
As usual, the bounded set is assumed Lebesgue measurable and an exponent is positive; the conventional Banach-space assertion concerns . No boundary regularity of is needed. Choose a smooth compactly supported cutoff equal to one on a ball containing . Weak convergence and the Uniform boundedness principle give a common bound for , hence also for . These functions have a common compact support.
We prove their local compactness directly. Translation and the fundamental theorem along line segments give, by density from smooth functions,
For a compactly supported mollifier , integration of this estimate gives
At fixed , Hölder's inequality bounds and uniformly by the norm of the corresponding smoothing kernels. Their supports lie in one fixed compact set, so the Arzelà-Ascoli theorem makes this smoothed family precompact in . The uniform approximation makes the original family totally bounded in : choose a finite net for the smoothed family and add the approximation error. Completeness gives relative compactness.
Every strongly convergent subsequence of has limit , since multiplication by preserves weak convergence. Consequently the whole sequence converges strongly: otherwise a subsequence with distance at least a fixed positive number would have a further convergent subsequence and contradict that unique limit. In particular .
The critical Sobolev inequality supplies a uniform bound in . For clarity, this estimate follows from part 2(a), not from compactness at the critical exponent. For a compactly supported smooth , integrating by parts against the vector field gives
Its distributional divergence is the sphere-area multiple of a point mass; applying the Hardy–Littlewood–Sobolev inequality with yields . Density extends this to .
For choose with . Hölder's inequality gives
For , use . The same integral estimate also works for if is understood with its usual quasi-norm. Thus strongly for every positive . We have never asserted that multiplying by the possibly nonsmooth indicator preserves .