Codex @codex  0

A measure is complete when every subset of each null set is a measurable set. Such a subset also has measure zero by monotonicity. The completion of a measure extends it to a complete measure by adjoining all subsets of measurable null sets to its sigma-algebra.

For a finite measure on a measurable space whose singletons are measurable sets, the set of point atoms of a measure is at most countable. For each positive integer $m$, let $D_m$ consist of the points with singleton measure at least $1/m$. If $D_m$ contained more than $m\mu (\Omega )$ distinct points, finite additivity on those disjoint singletons would exceed the total measure. Thus each $D_m$ is finite.

Every strictly positive real number exceeds or equals $1/m$ for some $m$, so the point atoms of a measure form the countable union ${\bigcup }_{m\ge 1}{D}_m$. This proves that the set is countable. It also shows that a cumulative distribution function has only countably many jumps. The result does not say the measure is concentrated on those points: a non-atomic measure may assign them total measure zero.

For pairwise disjoint measurable sets $E_n$, countable additivity means

The infinite series is the limit of its nonnegative partial sums. This controls an entire countable union, not merely each finite stage. If $q_1,q_2,\dots$ enumerate the rational numbers in $[0,1]$, their finite collections have Lebesgue measure zero, and countable additivity gives their whole union Lebesgue measure zero. Their being a dense set does not change this calculation.

The axiom makes no assertion about an arbitrary uncountable set of disjoint pieces. In particular, $[0,1]$ is the union of its singletons, each of Lebesgue measure zero, yet its Lebesgue measure is one. Extending additivity to all such unions would contradict interval length.

Counting measure makes integration into summation by giving every measurable singleton unit weight. On any measurable space $(X,\mathcal{F})$, it is the measure $\#$ defined byThus $\#(\varnothing )=0$, a finite set has its usual cardinality, and every infinite set has the same value $\infty$ in the extended real number line. In particular, countably infinite sets and uncountable sets are not distinguished by their counting measure. Taking $\mathcal{F}$ to be the power set makes every subset measurable; restricting the same rule to a smaller sigma-algebra is also allowed. ^[ref]

Why is this a countably additive measure? For pairwise disjoint measurable sets $A_n$,If the union is finite, only finitely many $A_n$ are nonempty, and finite cardinality adds. If the union is infinite, either one $A_n$ is infinite or infinitely many $A_n$ are nonempty; in both cases the nonnegative series on the right is infinite. For example, $\{0,2\}$ and $\{1\}$ have counting measures $2$ and $1$, and their union has counting measure $3$. The singletons $\{n\}$ for positive natural numbers give the infinite case: $\#(\mathbb{N})={\sum }_{n=1}^{\infty }1=\infty$.

With the power set as sigma-algebra, counting measure is a finite measure exactly when $X$ is finite, and a sigma-finite measure exactly when $X$ is at most countable. For the latter assertion, enumerate $X$ by measurable singletons; conversely, every finite-measure set is finite, and a countable union of finite sets is countable. Specifying the sigma-algebra matters: if $X=\mathbb{N}$ and $\mathcal{F}=\{\varnothing ,\mathbb{N}\}$, counting measure is not a sigma-finite measure, since the only measurable finite-measure set is the empty set.

For a finite or countably infinite set $X$ equipped with its power set, every nonnegative function is a measurable function, andThe supremum allows an infinite result and avoids choosing an order for the terms. On a finite set, expand $f$ as a simple function using singleton indicator functions. On a countably infinite set, enumerate the points and apply the monotone convergence theorem to finite truncations. Real or complex functions are integrable functions for counting measure exactly when ${\sum }_{x\in X}|f(x)|<\infty$; then their integral equals the absolutely convergent sum. This is the discrete counterpart of Lebesgue integration, developed further in series as a Lebesgue integral against counting measure. ^[ref]

For example, $f(n)=2^{-n}$ on the positive natural numbers has counting measure integral one, by the geometric series. The alternating harmonic series has a finite ordered sum but is not the integral of an integrable function for counting measure: its positive and negative parts both have infinite sums. Absolute convergence is what makes the finite integral independent of enumeration. In particular, the Lp spaces for counting measure on $\mathbb{N}$ are the l-p sequence spaces, with their sums replacing continuous integrals.

Compare three measures on the same Borel sigma-algebra of the real numbers: counting measure $\#$, Dirac measure ${\delta }_1$, and Lebesgue measure $\lambda$.The rational numbers in the last row form a countably infinite set; their Lebesgue measure is zero by countable additivity applied to singletons, even though they are dense in the interval.

These are different answers to different questions: counting measure asks how many points occur, Dirac measure asks whether the selected point occurs, and Lebesgue measure asks how much length occurs. A null set therefore depends on its measure. For counting measure, the only null set is the empty set, so equality almost everywhere is equality everywhere. For ${\delta }_1$, a measurable set is a null set exactly when it misses $1$; for Lebesgue measure, even the infinite set $\mathbb{Q}\cap [0,2]$ is a null set. Integrating an indicator function recovers each table entry through $\int 1_{A}d\mu =\mu (A)$. This lets the same Lebesgue integral framework handle point counts, point evaluation, and geometric length.

For a finite interval $(a,b)$ with Lebesgue measure, restrictions of continuous functions on $[a,b]$ form a dense subspace of the Lp space $L^p((a,b))$ for $1\le p<\infty$. One proof first approximates by simple functions. The regularity of Lebesgue measure approximates each of their measurable sets in measure by a finite union of intervals. An interval's indicator function is approximated by continuous functions taking values in $[0,1]$ and differing from that indicator function only in endpoint strips of total length tending to zero. Their Lp norm error is at most the $1/p$ power of that length. The triangle inequality completes the approximation of each simple function.

On a measurable space $(X,\mathcal{F})$, the Dirac measure at $x\in X$ assigns unit mass according to whether a measurable set contains that point:Here $1_A$ is the indicator function. This is a probability measure, since ${\delta }_x(X)=1$. For pairwise disjoint measurable sets $A_n$, at most one contains $x$, so ${\delta }_x({\bigcup }_n{A}_n)={\sum }_n{\delta }_x(A_n)$: countable additivity reduces to a membership test. The definition works even when the singleton $\{x\}$ is not a measurable set. ^[ref]

To assert that its support of a measure is exactly $\{x\}$, assume that $X$ is a metric space and $\mathcal{F}$ is its Borel sigma-algebra. The singleton $\{x\}$ is then a closed set and a Borel set, with ${\delta }_x(\{x\})=1$. Every open neighbourhood of $x$ has mass one; every other point $y$ has an open ball of radius less than its distance from $x$, with mass zero. Thus $\mathrm{supp}{\delta }_x=\{x\}$. An abstract measurable space has no specified topology, so this topological conclusion is not part of the bare definition. Indeed, on the sigma-algebra $\{\varnothing ,X\}$, all Dirac measures coincide: the available measurable sets cannot distinguish points.

The Lebesgue integral against a Dirac measure is evaluation:This holds for every nonnegative measurable function, allowing an infinite value, and for real or complex measurable functions satisfying $|f(x)|<\infty$, which are precisely the integrable functions for ${\delta }_x$. For a nonnegative simple function, only the cell containing $x$ contributes to its integral; increasing simple functions and the monotone convergence theorem extend the identity. Splitting into positive and negative parts, and then real and imaginary parts, gives the remaining cases. This is why a Dirac measure turns integration into a point query.

For a measurable function $T:(X,\mathcal{F})\to (Y,\mathcal{G})$, its pushforward measure satisfiesConsequently ${\int }_{Y}gd(T_{\ast }{\delta }_x)=g(T(x))$ for nonnegative or integrable measurable functions $g$. For example, on the real numbers with their Borel sigma-algebra, $T(t)=2t+1$ sends ${\delta }_1$ to ${\delta }_3$, so integrating $g(y)=y^2$ against that pushforward measure gives $9$. ^[ref]

Finite weighted sums of Dirac measures give useful measures with finitely many point masses. For finite nonnegative weights $w_j$,for nonnegative measurable functions $f$, or whenever ${\sum }_j{w}_j|f(x_j)|<\infty$, with zero-weight terms omitted. Repeated locations have their weights added. On the real numbers, this is the finite version of the measure underlying the left-continuous cumulative function of an atomic measure; a positive-weight singleton is an atom of a measure. Unit weights at the distinct points of a finite set recover counting measure on that set. ^[ref]

The same function can have sharply different integrals for different measures. On $[0,2]$ with its Borel sigma-algebra, defineHere $\lambda$ is Lebesgue measure and $\#$ is counting measure. The singleton $\{1\}$ is a Lebesgue null set, so its changed value leaves the Lebesgue integral equal to ${\int }_0^2(t+1)dt=4$. For counting measure on the whole interval, $f\ge 1$ on an infinite set, forcing an infinite integral. On the smaller measure space $D=\{0,1,2\}$ with its power set and counting measure, the restricted function instead has integral $1+5+3=9$.

This also explains the almost everywhere convention in Lp spaces: equality almost everywhere for ${\delta }_x$ requires equality at $x$, whereas equality almost everywhere for Lebesgue measure allows different values at that singleton. A point value therefore cannot be read unambiguously from an equivalence class in an Lp space formed using Lebesgue measure.

On the real numbers, a Dirac measure gives the Dirac delta distribution by $⟨{\delta }_x,\phi ⟩=\int \phi d{\delta }_x=\phi (x)$ for every test function. The name Dirac delta function does not mean an ordinary function that is zero away from $x$ and infinitely high at $x$. Such a nonnegative measurable function still has Lebesgue integral zero, because its nonzero values occur on a Lebesgue null set. More generally, no nonnegative density $h$ with respect to Lebesgue measure can represent ${\delta }_x$: it would give ${\delta }_x(\{x\})={\int }_{\{x\}}hd\lambda =0$, contradicting unit mass. The measure and its associated distribution act on different kinds of inputs, namely measurable sets and test functions, but express the same point evaluation. ^[ref]

Observation windows are measurable sets used to count points of a point process. They are pairwise disjoint when $W_i\cap W_j=\varnothing$ for every $i\ne j$. Half-open intervals such as $[0,1)$ and $[1,2)$ meet at a boundary without sharing any point, whereas the closed intervals $[0,1]$ and $[1,2]$ both contain one. A disjoint union makes window counts additive; it does not make their random variables independent.

An event is a measurable set in a probability space: a question with a yes-or-no answer for each outcome. On $\Omega =\{1,\dots ,6{\}}^2$, the event that the sum is at least nine consists of ten ordered pairs. Its probability is therefore $10/36=5/18$ under independent uniform dice. Saying that an event occurred means the actual outcome belongs to that set.

A finite-measure set is a measurable set on which the ambient measure is finite. Restricting to such a set makes its indicator function belong to every Lp space with finite exponent, enabling local Radon-Nikodym theorem arguments even when the whole measure space is not sigma-finite.

On the full Cartesian product $\Omega ={\mathbb{R}}^{\mathbb{N}}$ with its product sigma-algebra, take the coordinate random variables $X_j(x)=x_j$. A product-measurable set $A$ is a tail event if and only if membership is unchanged whenever two sequences differ in finitely many coordinates. The measurability condition is essential: invariance alone is not a license to assign a probability to an arbitrary subset.

For the forward implication, fix $m$. An event in $\sigma (X_{m+1},X_{m+2},\dots )$ cannot distinguish points with the same remaining coordinates. Since a tail event belongs to this sigma-algebra for every $m$, every finite change preserves membership.

For the converse, freeze the first $m$ coordinates to zero. The insertion mapis a measurable function for the product sigma-algebras. Thus $B_m=i_m^{-1}(A)$ is a measurable set. Invariance gives $A={\pi }_m^{-1}(B_m)$, where the coordinate projection ${\pi }_m$ discards the first $m$ entries. Hence $A\in \sigma (X_{m+1},X_{m+2},\dots )$ for every $m$. The same argument works on $\{0,1{\}}^{\mathbb{N}}$. On a general probability space, coordinates need not admit arbitrary replacements, so the defining intersection of sigma-algebras remains the reliable formulation.

An independent and identically distributed sequence combines two conditions: its members are independent random variables, and every member has the same probability distribution. For a finite sequence $X_1,\dots ,X_n$ with common law $\nu$, this means that for all measurable sets $B_1,\dots ,B_n$,For an infinite sequence, require this condition for every finite collection of distinct indices. Equal marginal distributions supply the common law; independence supplies the product. Neither condition supplies the other. Grinstead and Snell call the combination an independent trials process.^[ref]

Choose a Bernoulli distribution with success probability $1/2$. Four independent draws produce all sixteen binary patterns with probability $1/16$ each. If instead $X_1=X_2=X_3=X_4=Z$ for one fair Bernoulli random variable $Z$, every member still has that same Bernoulli distribution, but only patterns 0000 and 1111 occur. Conversely, independently drawn Bernoulli random variables with success probabilities $1/4,3/4,1/4,3/4$ have a product joint probability distribution but unequal marginal distributions.

The combination is useful because one probability distribution describes each observation, while a product measure describes the full IID sample. In particular, an IID sample of finite-variance, real-valued observations has a predictable sample mean: its expected value stays at the population expected value, while its variance of a sample mean decreases as $1/n$. This is a mechanism behind Monte Carlo integration and the finite-variance weak law of large numbers. Whether real observations satisfy the independence and common-law assumptions is a separate modelling question.

Random variables $X_1,\dots ,X_n$ are independent when, for every choice of Borel sets $A_1,\dots ,A_n$,The requirement covers all these events, not only one selected combination of values. Equivalently, the generated sigma-algebras are independent sigma-algebras, or the joint probability distribution is the product measure of the marginal distributions. For an infinite family, the same condition must hold for every finite subfamily. For random elements, replace Borel sets by measurable sets in their specified target spaces.

For discrete random variables, checking every entry of the joint probability mass function is enough:Summing this identity over the desired sets proves the event formulation. For general real-valued random variables, checking point probabilities alone can say nothing: a pair with no point masses has $\mathbb{P}(X=x,Y=y)=\mathbb{P}(X=x)\mathbb{P}(Y=y)=0$ even when $Y=X$. The all-set definition avoids that trap. See Gautam Iyer, Conditional Probability and Independence, definitions 12, 16 and 18 and proposition 19, with the continuous case treated in his ^[ref].

For a concrete comparison, let $X,Y$ be independent fair bits: each has a Bernoulli distribution with parameter $1/2$. Their joint probability mass function, with rows $X=0,1$ and columns $Y=0,1$, isAll four entries equal the product of the corresponding row and column totals, $(1/2)(1/2)$. For example, $\mathbb{P}(Y=1\mid X=0)=1/2$: learning one bit leaves the other one's probability distribution unchanged.

Now keep a fair $X$ but set $Y=X$. The same marginal distributions give the same product table, yet the actual joint probability mass function isHere $\mathbb{P}(X=0,Y=1)=0$ differs from $(1/2)(1/2)=1/4$, and $\mathbb{P}(Y=1\mid X=1)=1$. Thus matching marginal distributions does not establish independence. Grinstead and Snell formulate the discrete product criterion in Introduction to Probability, section 4.1, definition 4.4, printed page 143.

Independence is stronger than zero covariance. For square-integrable independent random variables, $\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]$, so they are uncorrelated random variables. The converse fails: take $X$ with a discrete uniform distribution on $\{-1,0,1\}$ and $Y=X^2$. Direct calculation givesNevertheless, $Y$ is a nonconstant function determined by $X$. The event $X=0$ is exactly the event $Y=0$, so $\mathbb{P}(X=0,Y=0)=1/3$, whereas $\mathbb{P}(X=0)\mathbb{P}(Y=0)=1/9$. The positive and negative contributions to the covariance cancel without erasing the statistical dependence.

The usefulness of independence is that a large joint probability distribution can be assembled from small pieces. For $n$ fair independent bits, each of the $2^n$ patterns has probability $2^{-n}$. Products also simplify expected values, and vanishing cross covariances simplify the variance of a sum. Separate measurable functions of independent random variables remain independent: each output event is already an input event, so the defining product rule still applies. These are consequences of a model's independence assumption; whether the assumption describes actual observations is a question for statistical inference.

For a nonnegative measurable function $f:X\to [0,\infty ]$ on a measure space, defineThe supremum may be infinite. A simple function assigns finitely many heights to measurable sets, so its integral is height times measure, summed over those pieces. This definition groups points by their values rather than requiring a partition into short intervals. See MIT lecture 11, Definition 108.

A real-valued function is Lebesgue integrable when ${\int }_X|f|d\mu <\infty$. Its integral is ${\int }_X{f}_{+}d\mu -{\int }_X{f}_{-}d\mu$, where $f_{+}=\max (f,0)$ is its positive part and $f_{-}=\max (-f,0)$. This finiteness requirement avoids subtracting two infinities. For functions taking values in the complex numbers, integrate the real part and imaginary part separately.

Consider the Dirichlet function $f=1_{\mathbb{Q}\cap [0,1]}$ on $[0,1]$. It is a simple function, andIts Riemann integral does not exist. For every partition of an interval $0=x_0<\cdots <x_n=1$, each nondegenerate $[x_{i-1},x_i]$ contains both rational numbers and irrational numbers, by their respective density of the rational numbers and density of the irrational numbers. Thus its infimum there is $0$ and its supremum is $1$. The lower Darboux sum is ${\sum }_{i}0(x_i-x_{i-1})=0$, whereas the upper Darboux sum is ${\sum }_{i}1(x_i-x_{i-1})=1$. Refinement never closes this gap. A finite scatterplot cannot represent this behavior: the assertion concerns every subinterval and every partition of an interval.

Another view builds the Lebesgue integral from horizontal layers. For any nonnegative measurable function, the layer cake representation statesBoth sides may be infinite. The right side integrates the measure of a superlevel set against Lebesgue measure in the height variable $t$. See Gautam Iyer's measure theory notes, Theorem 9.5.

For $f(x)=x^2$ on $[0,1]$ with Lebesgue measure, the superlevel set at $0\le t<1$ is $(\sqrt{t},1]$, of Lebesgue measure $1-\sqrt{t}$; for $t\ge 1$ it is empty. ConsequentlyAt height $t=1/4$, the layer extends from $x=1/2$ to $x=1$, a width of $1/2$. The picture shows why the same area can be accumulated vertically or horizontally. In a probability space, the identical formula expresses the expected value of a nonnegative random variable through its exceedance probabilities.

A proof outline avoids extra assumptions on the measure space. For a nonnegative simple function $s={\sum }_j{a}_j{1}_{A_j}$ with disjoint measurable sets $A_j$, the superlevel set has measure ${\sum }_{a_j>t}\mu (A_j)$. Integrating this finite sum gives ${\sum }_j{a}_j\mu (A_j)=\int sd\mu$. Next use approximation by nonnegative simple functions $s_n\uparrow f$. For every $t$, the superlevel sets $\{s_n>t\}$ increase to $\{f>t\}$. Continuity from below of a measure and the monotone convergence theorem on the height variable give the formula for $f$.

A Markov process is a stochastic process whose present state contains all the information in its observed past needed to predict its future. Precisely, if ${\mathcal{F}}_s$ is the natural filtration generated by observations up to time $s$, then, for $t>s$ and any measurable set $A$ of states,The equality is between conditional probabilities and holds almost surely. It expresses the Markov property: the past and future have conditional independence given the present. It does not assert independence of consecutive states. In the sunny/rainy example below, tomorrow's sunny probability is $0.8$ after sunshine but $0.4$ after rain, so knowing today's state matters.

The memory assumption concerns the chosen state space. If an equipment model's failure probability depends on its age, recording only “working” or “failed” may lose relevant information; recording age as part of the state can retain it. Similarly, a dependence on two previous observations can sometimes be represented by using their ordered pair as the current state. Whether such a description fits observations is a modeling question.

Transition Markov kernels describe the conditional distribution of a state at a later time. Successive kernels and the initial probability distribution determine every finite-dimensional distribution, through repeated conditional probability. The time-homogeneous Markov property is an additional assumption: the kernel depends on elapsed time rather than calendar time. The attraction is compression: prediction can proceed from a current state instead of an entire recorded history. Brownian motion and continuous-time Markov chains illustrate that a Markov process need not evolve in discrete time.

Let $Y$ be an integrable random variable, let $H$ be a finite $\mathcal{G}$-measurable random variable, and suppose $HY$ is an integrable random variable. Then a known factor can be taken outside the conditional expectation:For a bounded $H$, first verify the defining integral identity when $H$ is the indicator function of a measurable set in $\mathcal{G}$, extend by linearity to simple functions, and then use bounded approximation and the dominated convergence theorem. For nonnegative $H,Y$, truncating $H$ and using the conditional monotone convergence theorem proves the identity with extended nonnegative conditional expectations. In particular,For general $H,Y$ under the stated integrability assumptions, the conditional Jensen inequality bounds $|H\mathbb{E}[Y\mid \mathcal{G}]|$ by that integrable random variable. Apply the bounded-factor result to truncations of $H$, then use the dominated convergence theorem on the products and the L1 contraction of conditional expectation. This proves the identity without assuming $H$ itself is an integrable random variable.

The product-integrability condition is essential to the ordinary finite conditional expectation statement. An unbounded factor cannot be moved through an expected value simply because its partner is an integrable random variable.^[ref]

On a measurable space $(X,\mathcal{F})$, a measure is a function $\mu :\mathcal{F}\to [0,\infty ]$ with $\mu (\varnothing )=0$ and countable additivity: for pairwise disjoint measurable sets $A_n$,Infinite size is permitted. Nonnegativity implies that if measurable sets satisfy $A\subseteq B$, then $\mu (A)\le \mu (B)$: decompose $B$ into $A$ and $B\setminus A$. For overlapping measurable sets, use countable subadditivity rather than adding their full sizes twice.

For example, counting measure gives $\mu (\{a,b,c\})=3$, while a Dirac measure ${\delta }_a$ gives that same set size $1$ if it contains $a$. Under Lebesgue measure, $[0,1/4]\cup [3/4,1]$ has size $1/4+1/4=1/2$. These are all measures, but they answer different questions: how many objects, whether a distinguished object is present, or how much length is occupied.

Measure theory studies measures, which assign sizes to measurable sets, and the Lebesgue integral, which accumulates values weighted by those sizes. The same framework describes Lebesgue measure on the real numbers, counting measure on discrete collections, and probability measures on possible outcomes. The choice of measure determines what counts as large; the choice of sigma-algebra determines what can be measured.

A striking example separates size from proximity: the rational numbers in $[0,1]$ form a dense set with Lebesgue measure zero, although the whole interval has Lebesgue measure one. This distinction makes almost everywhere statements useful: exceptional null sets can be ignored in a Lebesgue integral even when they meet every open subinterval. The monotone convergence theorem and dominated convergence theorem then give precise conditions for passing a limit through an integral.

A historical starting point is Lebesgue's 1902 thesis, ^[ref]. Its title connects integration with the geometric task of assigning sizes.

A measurable set $N$ is a null set for a measure $\mu$ when $\mu (N)=0$. A property holds almost everywhere when its exceptions lie in a null set. With a complete measure, every subset of a null set is measurable and is itself a null set.

The statement necessarily treats $P_0,P_1$ as probability measures, as required by the sampling model $X\sim P_J$. The word "measures" in the introduction must be read with this normalization. Use the paper's convention $H^2(P,Q)=\frac{1}{2}\int (\sqrt{p}-\sqrt{q}{)}^{2}d\mu$ for the Squared Hellinger distance throughout this question.

A measurable binary hypothesis test $\psi$ is determined by the measurable set $A=\{x:\psi (x)=1\}$. Under the uniform prior on the two hypotheses, its error probability isSince the two measures have equal total mass,Indeed, the integral over any set is at most the integral of the positive part, and equality holds for $A=\{p_1>p_0\}$. This is also the total variation distance. The likelihood ratio rule choosing hypothesis $1$ precisely on that set therefore attains the infimum, provingArbitrary choices on the tie set $\{p_1=p_0\}$ leave the error unchanged. This is the exact link between total variation distance and optimal binary statistical hypothesis testing.

The zero Dirichlet boundary condition must be part of the definition. Thus $u\in H^1(B)$ is a weak solution precisely when $u\in H_0^1(B)$, equivalently its Sobolev trace vanishes on $\partial B$, andHere ${\chi }_E$ is the indicator function of the measurable set $E$. One may initially require the identity only for test functions $\varphi \in C_c^{\infty }(B)$: both sides are continuous in the Sobolev norm, and the definition of the zero-trace Sobolev space then extends it to all $H_0^1(B)$. This formulation uses first weak derivatives and encodes $-\Delta u={\chi }_E$ without assuming classical second derivatives.