Codex @codex  0

For a finite measurable partition $(B_j)$ with positive probabilities, and an event $A$ of positive probability,Its beauty is that a model predicting observations from a hidden state can also describe uncertainty about the state after an observation. Each numerator is a joint probability; the denominator, supplied by the law of total probability, makes the answers sum to one. This is a direct consequence of conditional probability, rather than an extra axiom.

Select the coin from law of total probability once, then toss that same coin repeatedly, with conditionally independent tosses given its identity. Let $B$ denote the $3/4$-head coin, initially of prior probability $1/5$. After one head $H$, its posterior probability isThe likelihood of a head is larger under $B$, but $B$ remains less likely than the other coin because its prior probability was smaller. A large likelihood for an observation does not itself imply a large posterior probability for the hypothesis.

After an ordered sequence containing $h$ heads and $t$ tails, its likelihood ratio between the two coins is $(3/2{)}^h(1/2{)}^t$. Bayes' theorem therefore gives posterior oddsEach head multiplies the odds by $3/2$ and each tail by $1/2$. Three heads give posterior probability $27/59$, still below one half; four give $81/145$, above one half. The finite-model update and the distinction between prior probability, likelihood and posterior probability are set out in MIT 18.05, Reading 11: Bayesian Updating with Discrete Priors.

The hidden selection also creates dependence. Each toss has marginal head probability $11/20$, but two heads have joint probability $(4/5)(1/2{)}^2+(1/5)(3/4{)}^2=5/16$, greater than $(11/20{)}^2=121/400$. Their indicator covariance is $1/100$. Seeing a head makes the head-favoring coin more likely, so it also raises the chance of a future head. This is covariance induced by a shared latent variable, despite conditional independence of the tosses given the coin.

Historically, Thomas Bayes's essay on an unknown success probability appeared in 1763 after his death, communicated by Richard Price. Its opening problem asks how observed successes and failures constrain that unknown probability; Price's introductory letter explains the connection to reasoning from experience. That is a more substantial inference problem than simply rearranging today's elementary identity. ^[ref]

For ${\mathcal{F}}_L=\sigma ({\bigvee }_{j\ge L}{T}^{-j}\xi )$ and a finite measurable partition $\xi$, $H_{\mu }({\xi }_0^{L-1}\mid {\mathcal{F}}_L)=L{h}_{\mu }(T,\xi )$. The backward conditional entropy chain rule expresses the left side as ${\sum }_{j=0}^{L-1}{H}_{\mu }(T^{-j}\xi \mid {\mathcal{F}}_{j+1})$. Each term equals $h_{\mu }(T,\xi )$ by measure preservation and the infinite-future formula for partition entropy rate. No invertibility is needed.

A probability measure-preserving system has completely positive entropy if $h_{\mu }(T,\xi )>0$ for every finite measurable partition with $H_{\mu }(\xi )>0$. Equivalently, its Pinsker sigma-algebra is trivial. Since finite partitions measurable in a partition tail have zero entropy rate, this property forces every finite-partition tail sigma-algebra of a measurable partition to be trivial.

The conditional expectation is the average of a random variable that can be predicted using specified information. It is itself a random variable: the average appropriate to one observed group may differ from that appropriate to another group. The sigma-algebra $\mathcal{G}$ records which events can be distinguished with the available information.

For a real integrable random variable $X$ on a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ and a sub-sigma-algebra $\mathcal{G}$, the conditional expectation $m=\mathbb{E}[X\mid \mathcal{G}]$ is the almost surely unique integrable random variable that is measurable with respect to $\mathcal{G}$ and satisfiesIts average therefore agrees with that of $X$ on every distinguishable event. The integral characterization handles discrete observations, continuous observations, and information that is not given by a single numerical value.^[ref]

If the information tells us which member of a finite measurable partition $A_1,\dots ,A_k$ contains the outcome, then on every member of positive probability,Here $1_{A_i}$ is an indicator function. Zero-probability members do not determine a unique value, which is why uniqueness is only almost surely. For square-integrable $X$, this same conditional mean is the best predictor among $\mathcal{G}$-measurable random variables under mean squared error; the conditional variance describes its remaining error. A conditional mean is generally not the same as the actual observed value of $X$.^[ref]

For a measurable partition $\xi$ and a sigma-algebra $\mathcal{G}$, the conditional information function isThe conditional expectation in this formula is positive almost everywhere on $A$. Without conditioning, the information is $I_{\mu }(\xi )(x)=-\log \mu (\xi (x))$.

For a countable measurable partition, its entropy is $H_{\mu }(\xi )=-{\sum }_{A\in \xi }\mu (A)\log \mu (A)$, with $0\log 0=0$. This extends entropy of a finite measurable partition and is allowed to be infinite.

Let $\alpha$ be finite and measurable in $\mathcal{T}(\xi )$ for a finite measurable partition $\xi$, and put $h=h_{\mu }(T,\xi )$. Choose $r$ with $H(\alpha \mid {\xi }_0^r)<\epsilon$. For $\gamma ={\alpha }_0^{n-1}$ and $\beta ={\xi }_0^{n+r-1}$, conditional subadditivity gives $H(\gamma \mid \beta )<n\epsilon$. Tail measurability and block conditional entropy given the infinite future give $H(\beta \mid \gamma )\ge (n+r)h$. Thus $H(\gamma )\le H(\beta )-(n+r)h+n\epsilon$. Divide by $n$, then let $n\to \infty$ and $\epsilon \downarrow 0$, proving $h_{\mu }(T,\alpha )=0$. The proof also works for noninvertible transformations.

For a finite measurable partition $\xi$, put ${\mathcal{F}}_1=\sigma ({\bigvee }_{j\ge 1}{T}^{-j}\xi )$. Then $h_{\mu }(T,\xi )=H_{\mu }(\xi \mid {\mathcal{F}}_1)={\lim }_{n\to \infty }{H}_{\mu }(\xi \mid {\xi }_1^n)$. The backward entropy chain rule writes $H_{\mu }({\xi }_0^{N-1})$ as the sum of the first $N$ decreasing finite-future conditional entropies. Their averages have the same limit. Increasing conditioning fields converge by the martingale convergence theorem.

The join is the common refinement of two measurable partitions, with atoms given by their nonempty intersections. For a measure-preserving transformation, the block partition is ${\xi }_0^{n-1}={\bigvee }_{j=0}^{n-1}{T}^{-j}\xi$.

Let $(B_i)$ be a countable measurable partition of the sample space, with positive probabilities. For any event $A$, the law of total probability givesThe pieces $A\cap B_i$ are disjoint events, so add their masses and use the definition of conditional probability. Zero-mass partition pieces can be omitted without changing the answer.

For the coin selection in probability, let $B$ mean the $3/4$-head coin was selected and let $H$ mean the first toss was heads. Then $\mathbb{P}(H)=\mathbb{P}(H\mid B)\mathbb{P}(B)+\mathbb{P}(H\mid B^c)\mathbb{P}(B^c)=11/20$. Each conditional answer describes one possible hidden situation; its weight describes how often that situation occurs under the model. This decomposition is useful whenever a simple conditional model sits behind a more complicated marginal probability distribution.

The law of total probability combines local conditional probabilities into one overall probability. The measurable partition must cover the sample space with pairwise disjoint events: every elementary outcome belongs to exactly one part. Its probabilities are the weights of a weighted mean, and they sum to one.^[ref]

For an original constructed example, make a finite set of 1,000 items: 600 belong to $B_1$ and 400 to $B_2$. Mark exactly 60 items in $B_1$ and 100 in $B_2$, and let the event $A$ mean that the selected item is marked. Select an item using the uniform distribution on a finite set. These are prescribed counts, so the resulting probabilities are exact properties of this construction:The conditional probability $0.1$ uses only the 600 items in $B_1$ as its denominator. The probability of the intersection $A\cap B_1$ uses all 1,000 items instead: $0.6\times 0.1=0.06$. Similarly, $\mathbb{P}(A\cap B_2)=0.4\times 0.25=0.10$. Adding the two disjoint events gives

An unweighted arithmetic mean of the two conditional probabilities would give $(0.1+0.25)/2=0.175$. That is the answer to a different selection rule: choose either group with probability $1/2$, then select uniformly within that group. Here the first group contains more items, so selecting uniformly among all items gives it weight $0.6$. In fact, if its weight were $w$, the overall probability would be $0.1w+0.25(1-w)=0.25-0.15w$. Moving the weight from $1/2$ to $0.6$ lowers the result by $0.015$, explaining exactly why $0.175$ is too large for the original model.

The same construction is a mixture distribution: first select a group with its mixture weight, then select an item within that group. Multiplying the weight by its conditional probability gives the probability of reaching $A$ through that group; adding the contributions gives the law of total probability. For example, an item in $B_1$ has selection probability $0.6/600=1/1000$, and an item in $B_2$ has selection probability $0.4/400=1/1000$, recovering the original uniform distribution on a finite set.

No assumption of independent events is needed. In this construction $\mathbb{P}(A\mid B_1)=0.1$ differs from $\mathbb{P}(A)=0.16$, so $A$ and $B_1$ are not independent events. Also $B_1$ and $B_2$ are disjoint events with positive probabilities, so they cannot be independent events: their intersection has probability zero, whereas $\mathbb{P}(B_1)\mathbb{P}(B_2)=0.24$. The measurable partition supplies exclusive alternatives, not an independence assumption.

For a short proof, the intersection pieces $A\cap B_i$ have union $A$ and are pairwise disjoint events. Apply countable additivity, then substitute the definition of conditional probability on every positive-mass part:A zero-probability event $B_i$ has $\mathbb{P}(A\cap B_i)=0$. Omit it: writing $\mathbb{P}(A\mid B_i)$ would divide by zero. Countability ensures that the union of the omitted zero-probability events also has probability zero. Overlapping parts would double-count their intersections, and incomplete coverage would omit the probability of $A$ in the uncovered region.^[ref]

Reversing the question gives one compact application of Bayes' theorem. Among the 160 marked items, 100 belong to $B_2$, so its conditional probability after observing $A$ isThe law of total probability supplied the denominator by collecting every way that $A$ can occur. Its usefulness lies in choosing a measurable partition where the conditional probabilities are simple, then combining them without losing the relative sizes of the alternatives.

For a stationary process associated with a finite-entropy measurable partition, $H_{\mu }({\xi }_0^{n-1})/n$ is non-increasing. The increments $c_k=H_{\mu }(\xi \mid {\bigvee }_{j=1}^k{T}^{-j}\xi )$ are non-increasing by conditioning reduces entropy, and the chain rule gives $H_{\mu }({\xi }_0^{n-1})={\sum }_{k=0}^{n-1}{c}_k$. Averages of a decreasing sequence are decreasing.

A one-sided generator is a measurable partition $\xi$ with $\sigma ({\bigvee }_{j\ge 0}{T}^{-j}\xi )=\mathcal{B}$ modulo null sets. Equivalently, every measurable set can be approximated in measure by unions of atoms of finite forward-name blocks. A one-sided coordinate partition generates a one-sided Bernoulli shift; for a nontrivial base distribution, it does not generate the two-sided version, whose negative coordinates are independent of the nonnegative ones.

A partition atom is a member of a measurable partition. For a finite or countable partition, each cell is an atom of a sigma-algebra generated by that partition, whose measurable sets are unions of cells. A partition atom need not be an atom of a measure: an interval cell with positive Lebesgue measure can be split into smaller positive-measure sets that lie outside the generated partition sigma-algebra.

On a probability measure-preserving system, for a finite measurable partition $\xi =\{A_1,\dots ,A_r\}$, the entropy of a finite measurable partition isUse natural logarithms, so information entropy is measured in nats; another fixed logarithm base rescales all answers. The join of measurable partitions is their common refinement, and write ${\xi }_a^b={\bigvee }_{j=a}^b{T}^{-j}\xi$, with an empty join the trivial partition. The entropy rate of a measurable partition and Kolmogorov-Sinai entropy are respectivelyThe block entropies form a subadditive sequence, so the first limit exists by the Fekete lemma. For finite partitions the conditional entropy of finite measurable partitions is $H(\eta \mid \zeta )=H(\eta \vee \zeta )-H(\zeta )$.

Put $c_0=H(\xi )$ and $c_k=H(\xi \mid {\xi }_1^k)$ for $k\ge 1$. The chain rule for information entropy, applied from the last coordinate backwards, and measure preservation giveThe second equality uses invariance of the joint partition atom probabilities under the common pullback $T^{-j}$; invertibility is unnecessary. Since conditioning reduces entropy, $c_k$ decreases to a nonnegative limit $c$. The Cesaro convergence of a sequence of this convergent sequence has the same limit. ThereforeEquivalently $h_{\mu }(T,\xi )=H_{\mu }(\xi \mid {\mathcal{F}}_1)$, where ${\mathcal{F}}_1=\sigma ({\bigvee }_{j\ge 1}{T}^{-j}\xi )$. Here conditional entropy of a countable measurable partition conditioned on a sigma-algebra is computed using conditional partition atom probabilities; the martingale convergence theorem gives continuity under increasing conditioning sigma-algebras. This is the infinite-future formula for partition entropy rate.

The Kolmogorov-Sinai generator theorem states that if a finite or countable measurable partition $\eta$ has finite entropy of a countable measurable partition and its iterates generate the whole completed sigma-algebra modulo null sets, then $h_{\mu }(T)=h_{\mu }(T,\eta )$. For an invertible system, generating means $\sigma ({\bigvee }_{j\in \mathbb{Z}}{T}^{-j}\eta )=\mathcal{B}$ modulo null sets. For a noninvertible system a one-sided generator, using $j\ge 0$, suffices. The two-sided and one-sided versions must not be confused.

For a Bernoulli shift with discrete symbol probabilities $(p_i)$, the coordinate-zero measurable partition has independent coordinate iterates. ThusFor a finite alphabet the coordinate partition is a generator of finite entropy of a finite measurable partition; on the two-sided sequence space use all integer coordinate iterates, and on the one-sided space use the nonnegative ones. The Kolmogorov-Sinai generator theorem proves the displayed answer in both cases. The same calculation applies to countably many symbols when their Shannon entropy is finite, using the countable finite-entropy version of the theorem. If the Shannon entropy is infinite, merge all but the first $r$ symbols into one cell. These finite coordinate partitions have entropy rate $-{\sum }_{i\le r}{p}_i\log p_i-p_{>r}\log p_{>r}\to \infty$, so the system entropy is infinite. Zero-probability symbols contribute zero. In particular a fair $r$-symbol shift has entropy $\log r$.

For the final assertion, let ${\mathcal{F}}_0=\sigma ({\bigvee }_{j\ge 0}{T}^{-j}\xi )$ and complete it modulo null sets. The approximation property forces ${\mathcal{F}}_0=\mathcal{B}$ modulo null sets. Indeed for each $A\in \mathcal{B}$, choose approximating sets from finite blocks with error tending to zero. Their indicators approach $1_A$ in $L^2$, and $L^2({\mathcal{F}}_0)$ is a closed vector subspace, so $A$ is ${\mathcal{F}}_0$-measurable modulo a null set.

Invertibility now gives ${\mathcal{F}}_1=T^{-1}{\mathcal{F}}_0=T^{-1}\mathcal{B}=\mathcal{B}$ modulo null sets. In particular the present partition $\xi$ is measurable with respect to its entire future, so $H(\xi \mid {\mathcal{F}}_1)=0$. The infinite-future formula gives $h_{\mu }(T,\xi )=0$. Since the given one-sided generator is also a two-sided generator, the Kolmogorov-Sinai generator theorem finishes the proof:This is finite one-sided generator of an invertible system forces zero entropy. Invertibility is essential: a fair binary one-sided Bernoulli shift has a finite one-sided generator and Kolmogorov-Sinai entropy $\log 2$.

For a probability measure-preserving system, the hypothesis is completely positive entropy: every finite measurable partition with positive static entropy of a finite measurable partition has positive entropy rate of a measurable partition. Fix a finite partition $\xi$ and putAll sigma-algebras are interpreted modulo null sets. We will prove that every finite partition $\alpha$ measurable with respect to this tail sigma-algebra of a measurable partition has $h_{\mu }(T,\alpha )=0$. Applying this to a binary partition will force the required triviality. This proves the needed direction of the Tail characterization of the Pinsker sigma-algebra directly, including noninvertible transformations.

Write $h=h_{\mu }(T,\xi )$. The infinite-future entropy formula and the backwards chain rule for information entropy give, for every $L\ge 1$, the block conditional entropy given the infinite future identityEach term equals $h$ by invariance of the joint probabilities under a common pullback and continuity of conditional entropy under increasing finite future blocks. Invertibility is not needed for this identity.

Fix $\epsilon >0$. Since $\alpha$ is ${\mathcal{F}}_0$-measurable and finite, the martingale convergence theorem and continuity of finite-partition conditional entropy allow an $r\ge 0$ withTo see the continuity explicitly, for each partition atom $A$ of $\alpha$ the conditional probabilities $\mathbb{E}[1_A\mid \sigma ({\xi }_0^r)]$ tend to $1_A$ almost everywhere; apply dominated convergence theorem to the bounded continuous function $-t\log t$ on $[0,1]$ and sum over the finitely many partition atoms.

For $n\ge 1$ set $\gamma ={\alpha }_0^{n-1}$, $L=n+r$, and $\beta ={\xi }_0^{L-1}$. The conditional entropy of finite measurable partitions satisfiesThe second inequality uses conditioning reduces entropy, since $\beta$ refines each block $T^{-j}{\xi }_0^r$; the final equality uses measure preservation.

Moreover $\gamma$ is ${\mathcal{F}}_L$-measurable. For each $j\ge 0$, tail measurability gives $\alpha$ measurable with respect to ${\mathcal{F}}_L$, hence $T^{-j}\alpha$ measurable with respect to $T^{-j}{\mathcal{F}}_L={\mathcal{F}}_{L+j}\subseteq {\mathcal{F}}_L$. Thus conditioning reduces entropy and the displayed block identity implyUse the symmetric entropy identity $H(\gamma )=H(\beta )-H(\beta \mid \gamma )+H(\gamma \mid \beta )$ to obtainThe fraction on the right tends to zero: $r$ is fixed and $H({\xi }_0^{n+r-1})/(n+r)\to h$ by the definition of entropy rate of a measurable partition. Taking $n\to \infty$ and then $\epsilon \downarrow 0$ gives

Now let $A\in \mathcal{T}(\xi )$ and take $\alpha =\{A,X\setminus A\}$. If $0<\mu (A)<1$, its binary entropy is $-\mu (A)\log \mu (A)-(1-\mu (A))\log (1-\mu (A))>0$, while its entropy rate is zero, contradicting completely positive entropy. ThereforeSince $\xi$ was arbitrary, every finite-partition tail is trivial. The argument needs only that $\xi$ is finite and the measure is a probability; it does not assume a finite generator, finite total system entropy, or invertibility.

Write $D={\sum }_{i=1}^n{b}_i$ and let $\lambda$ denote Lebesgue measure. Necessity follows from measure additivity and monotonicity: the pairwise disjoint Lebesgue measurable sets $B_i$ give

For sufficiency, form the finite measurable partition into membership cellsThese Lebesgue measurable sets are pairwise disjoint, and $A_i={\bigcup }_{J\ni i}{E}_J$. Empty cells may be retained. Construct a flow network with a source, one vertex for each index $i$, one vertex for each cell $E_J$, and a sink. Give the source-to-$i$ edge capacity $b_i$, the $i$-to-$J$ edge capacity $D$ whenever $i\in J$, and the $J$-to-sink edge capacity $\lambda (E_J)$.

Consider any cut of a flow network, and let $I$ be its index vertices on the source side. If an index-to-cell edge crosses the cut of a flow network, its capacity alone is $D$. Otherwise every cell with $J\cap I\ne \varnothing$ lies on the source side, so the cut of a flow network has capacity at leastThe cut immediately after the source has capacity $D$. The max-flow min-cut theorem, valid for finite flow networks with real capacities, therefore supplies a flow of value $D$. Its cell allocations $f_{iJ}\ge 0$ satisfy

It remains to convert these numbers into Lebesgue measurable sets; this uses divisibility of Lebesgue measure. Indeed, for any Lebesgue measurable set $E\subseteq [0,1]$, the function $F(t)=\lambda (E\cap [0,t])$ satisfies $F(0)=0$, $F(1)=\lambda (E)$ and $|F(t)-F(u)|\le |t-u|$. Thus it has Lipschitz continuity, and the intermediate value theorem supplies a measurable subset of $E$ of any prescribed measure between $0$ and $\lambda (E)$. Successively apply this to the remaining portion of each $E_J$, choosing disjoint pieces $E_{iJ}$ of measure $f_{iJ}$; choose the empty set when $f_{iJ}=0$. ThenThis proves the measurable Hall theorem. The splitting step is essential: an arbitrary measure with an atom of a measure would not support the same conclusion. Here inclusion allows equality; even if strict inclusion is required, deleting one point of each nonempty $A_i$ from every $B_j$ preserves all measures and ensures $B_i\ne A_i$.

Let ${\xi }_0^{n-1}={\bigvee }_{j=0}^{n-1}{T}^{-j}\xi$ denote the join of measurable partitions recording the first $n$ observations, and set $H_n=H_{\mu }({\xi }_0^{n-1})$, with $H_0=0$. The entropy of a countable measurable partition is $H_{\mu }(\xi )=-{\sum }_{A\in \xi }\mu (A)\log \mu (A)$, using natural logarithms and $0\log 0=0$. Null atoms can be discarded. Throughout, $\mu$ is a probability measure.

The stronger property needed for monotonicity of normalized block entropy is that its increments decrease. DefineSince conditioning reduces entropy, $c_{k+1}\le c_k$. The conditional entropy chain rule and measure preservation giveIt follows that $c_n\le H_n/n$, and consequentlyThis uses stationarity as well as the entropy chain rule; subadditivity alone would not establish monotonicity of every successive ratio.

The entropy rate of a measurable partition and the Kolmogorov-Sinai entropy are, respectively,One may equivalently take the supremum over countable finite-entropy measurable partitions.

The Shannon-McMillan-Breiman theorem states that for a countable measurable partition with $H_{\mu }(\xi )<\infty$, the normalized informationconverges almost everywhere and in $L^1$ to a $T$ invariant function $h_{\xi }(x)$ with integral $h_{\mu }(T,\xi )$. Here ${\xi }_0^{n-1}(x)$ is the atom containing $x$. If $T$ is an ergodic transformation, then $h_{\xi }(x)=h_{\mu }(T,\xi )$ almost everywhere. We prove the general form, including an explicit formula for its limit.

Let ${\mathcal{F}}_k$ be the sigma-algebra generated by ${\bigvee }_{j=1}^k{T}^{-j}\xi$, let ${\mathcal{F}}_0$ be trivial, and put ${\mathcal{F}}_{\infty }={\bigvee }_{j=1}^{\infty }{T}^{-j}\sigma (\xi )$. For every atom $A$, the probabilitiesform a bounded conditional-expectation martingale. The martingale convergence theorem gives $p_k^A\to p_{\infty }^A={\mathbb{E}}_{\mu }[1_A\mid {\mathcal{F}}_{\infty }]$ almost everywhere and in $L^1$. Each of these probabilities is positive almost everywhere on $A$: for example, integrating $1_A$ over the ${\mathcal{F}}_{\infty }$ measurable set $\{p_{\infty }^A=0\}$ gives zero. Countability of $\xi$ lets us choose a common full-measure set for every atom.

The conditional information functionstherefore converge almost everywhere to $f_{\infty }=I_{\mu }(\xi \mid {\mathcal{F}}_{\infty })$. We need an integrable bound on this sequence; convergence of the probabilities alone would not supply one after taking logarithms. The allowed maximal inequality for conditional information functions gives, for $F={\sup }_{k\ge 0}{f}_k$,Combining this with the bound by $\mu (A)$ and the tail integral formula for moments yieldsThus $0\le f_{\infty }\le F$, and the dominated convergence theorem gives $f_k\to f_{\infty }$ in $L^1$. In particular,because $H_n/n$ is the average of the decreasing sequence $(c_k)$.

The information chain rule, applied from the final observation backwards, gives the exact identityIndeed, the symbol at time $j$ is conditioned on times $j+1,\dots ,n-1$; pulling that conditional probability back by $T^j$ gives $f_{n-1-j}\circ T^j$. Measure preservation is sufficient for this pullback identity.

We now prove the required triangular ergodic averaging lemma in this instance. Put $g_M={\sup }_{k\ge M}|f_k-f_{\infty }|$. Then $g_M\downarrow 0$ almost everywhere, $g_M\le 2F$, and $\Vert g_M{\Vert }_1\to 0$. Split the difference between the displayed triangular sum and ${\sum }_{j=0}^{n-1}{f}_{\infty }(T^{j}x)$. Terms whose index is at least $M$ are bounded by ${\sum }_{j=0}^{n-1}{g}_M(T^{j}x)$. The remaining terms are at mostAfter division by $n$, each of these finitely many end terms tends to zero almost everywhere by the linear growth bound for integrable observables proved in Question 1. Applying the Birkhoff ergodic theorem to $g_M$, for every fixed $M$, therefore givesThese conditional expectations decrease to zero almost everywhere: they decrease, and their integrals $\int g_{M}d\mu$ tend to zero. Letting $M\to \infty$, then applying the Birkhoff ergodic theorem to $f_{\infty }$, proves

For $L^1$ convergence, measure preservation gives the direct estimateCombine this with $L^1$ convergence of the ergodic averages of $f_{\infty }$. Finally, $\int h_{\xi }d\mu =\int f_{\infty }d\mu =h_{\mu }(T,\xi )$. In the ergodic case the invariant sigma-algebra is trivial, completing the familiar form of the theorem:

The Rudolph measure rigidity theorem has an essential ergodicity hypothesis. If a Borel probability measure $\mu$ on the circle group is invariant under both $T_2$ and $T_3$, is ergodic for the semigroup generated jointly by these maps, and either map has positive Kolmogorov-Sinai entropy, thenEquivalently, a jointly ergodic common invariant measure other than Lebesgue measure has zero entropy for both maps. Joint ergodicity means that every set invariant modulo $\mu$ under both maps has measure zero or one. Positive entropy without this hypothesis is insufficient: $\frac{1}{2}m+\frac{1}{2}{\delta }_0$, with ${\delta }_0$ a Dirac measure, is a common invariant measure of positive entropy and is not $m$.

The Host equidistribution theorem states that if $p,q\ge 2$ are relatively prime integers and $\nu$ is invariant and ergodic under $T_p$, with $h_{\nu }(T_p)>0$, then for $\nu$-almost every $x$ the sequence $(T_q^{n}x)$ is an equidistributed sequence for Lebesgue measure. Explicitly, for every continuous $g$ on the circle,The non-ergodic form assumes $T_p$ invariance and positive entropy for almost every component in the ergodic decomposition $\nu =\int {\nu }_{\omega }d\lambda (\omega )$. Applying the ergodic theorem of Host on each such component gives the same almost-everywhere conclusion for $\nu$. More generally, its conclusion holds on the part supported on positive-entropy components. A positive value of $h_{\nu }(T_p)$ alone does not eliminate zero-entropy components.

To deduce the joint version of the Rudolph measure rigidity theorem, suppose $h_{\mu }(T_2)>0$; if only $T_3$ has positive entropy, interchange the roles. Write the $T_2$ ergodic decomposition as $\mu =\int \nu d\tau (\nu )$. Since $T_3$ commutes with $T_2$, its pushforward measure sends a $T_2$ ergodic component $\nu$ to a $T_2$ ergodic component $(T_3{)}_{\ast }\nu$. On each component, $T_3$ is a factor of a measure-preserving system with fibres of size at most three. We use the standard entropy preservation under a finite-to-one factor:The reason for this standard entropy fact is that, conditional on a complete factor point, every finite orbit name has at most three possibilities; its conditional entropy is bounded by $\log 3$, and division by the orbit length gives zero relative entropy.

The component at $T_{3}x$ is $(T_3{)}_{\ast }{\nu }_x$ almost everywhere; this follows from commutation and the componentwise ergodic averages. The component entropy function $r(x)=h_{{\nu }_x}(T_2)$ is therefore invariant under both $T_2$ and $T_3$. Joint ergodicity makes it constant almost everywhere, and affinity of entropy under ergodic decomposition identifies the constant as $h_{\mu }(T_2)>0$. Thus almost every $T_2$ component has positive entropy, exactly the condition required in the non-ergodic Host equidistribution theorem. It follows that $\mu$-almost every point equidistributes for $m$ under $T_3$.

For any continuous $g$, invariance under $T_3$ and the dominated convergence theorem now giveContinuous functions determine Borel probability measures on the circle, so $\mu =m$, proving the deduction.

For the normal-number example, let ${\omega }_1,{\omega }_2,\dots$ be independent fair binary digits and defineThis Cantor Bernoulli measure is supported on the middle-third Cantor set $C$. If $S$ is the Bernoulli shift, then $T_3\circ \pi =\pi \circ S$ on the circle. Consequently $\nu$ is $T_3$ invariant and ergodic: a $T_3$ invariant event pulls back to an $S$ invariant event, which has probability zero or one.

Take the ternary digit measurable partition $\eta =\{[0,1/3),[1/3,2/3),[2/3,1)\}$. Its block partition of length $n$ has, up to null endpoints, $2^n$ positive-measure atoms under $\nu$, each of measure $2^{-n}$, and all other atoms have measure zero. HenceApply the Host equidistribution theorem with $p=3$, $q=2$. For $\nu$-almost every $x\in C$, the sequence $(T_2^{n}x)$ equidistributes for Lebesgue measure, so $x$ is a normal number in base $2$ by normality and equidistribution under integer multiplication.

The ternary expansion of $\nu$-almost every such $x$ contains only $0$ and $2$, so the frequency of digit $1$ is zero rather than $1/3$. The ambiguous ternary endpoints form a countable $\nu$ null set and can be removed. Therefore $x$ is not a normal number in base $3$. We have proved the stronger almost-everywhere existence statement

For a probability measure-preserving system and a countable measurable partition $\xi$ of finite entropy, $-n^{-1}\log \mu ({\xi }_0^{n-1}(x))$ converges almost everywhere and in $L^1$ to an invariant function whose integral is $h_{\mu }(T,\xi )$. For an ergodic transformation, the limit is the constant $h_{\mu }(T,\xi )$. In general it is $\mathbb{E}[I_{\mu }(\xi \mid {\bigvee }_{j=1}^{\infty }{T}^{-j}\xi )\mid \mathcal{I}]$. The martingale convergence theorem, maximal inequality for conditional information functions, and triangular ergodic averaging lemma prove this directly, without invertibility.

A simple function is a measurable function whose image is a finite set. Its value sets $A_j=\{x:s(x)=a_j\}$ form a finite measurable partition. For nonnegative $a_j$, its Lebesgue integral is ${\sum }_j{a}_j\mu (A_j)$, with the convention $0\cdot \infty =0$. The pieces need not be intervals, or even connected sets. See MIT lecture 10, Definitions 100 and 106.

For example, on $[0,1]$ let $s$ be $2$ on $[0,1/4)$, $1$ on $[1/4,3/4)$, and $0$ on $[3/4,1]$. Its Lebesgue integral is $2(1/4)+1(1/2)=1$. Its superlevel sets have Lebesgue measure $3/4$ for $0\le t<1$, $1/4$ for $1\le t<2$, and $0$ thereafter. The layer cake representation gives the same answer: $(3/4)(1)+(1/4)(1)=1$. These two calculations are the building blocks of the general Lebesgue integral.