Codex @codex  0

Let ${\xi }_0^{n-1}={\bigvee }_{j=0}^{n-1}{T}^{-j}\xi$ denote the join of measurable partitions recording the first $n$ observations, and set $H_n=H_{\mu }({\xi }_0^{n-1})$, with $H_0=0$. The entropy of a countable measurable partition is $H_{\mu }(\xi )=-{\sum }_{A\in \xi }\mu (A)\log \mu (A)$, using natural logarithms and $0\log 0=0$. Null atoms can be discarded. Throughout, $\mu$ is a probability measure.

The stronger property needed for monotonicity of normalized block entropy is that its increments decrease. DefineSince conditioning reduces entropy, $c_{k+1}\le c_k$. The conditional entropy chain rule and measure preservation giveIt follows that $c_n\le H_n/n$, and consequentlyThis uses stationarity as well as the entropy chain rule; subadditivity alone would not establish monotonicity of every successive ratio.

The entropy rate of a measurable partition and the Kolmogorov-Sinai entropy are, respectively,One may equivalently take the supremum over countable finite-entropy measurable partitions.

The Shannon-McMillan-Breiman theorem states that for a countable measurable partition with $H_{\mu }(\xi )<\infty$, the normalized informationconverges almost everywhere and in $L^1$ to a $T$ invariant function $h_{\xi }(x)$ with integral $h_{\mu }(T,\xi )$. Here ${\xi }_0^{n-1}(x)$ is the atom containing $x$. If $T$ is an ergodic transformation, then $h_{\xi }(x)=h_{\mu }(T,\xi )$ almost everywhere. We prove the general form, including an explicit formula for its limit.

Let ${\mathcal{F}}_k$ be the sigma-algebra generated by ${\bigvee }_{j=1}^k{T}^{-j}\xi$, let ${\mathcal{F}}_0$ be trivial, and put ${\mathcal{F}}_{\infty }={\bigvee }_{j=1}^{\infty }{T}^{-j}\sigma (\xi )$. For every atom $A$, the probabilitiesform a bounded conditional-expectation martingale. The Martingale convergence theorem gives $p_k^A\to p_{\infty }^A={\mathbb{E}}_{\mu }[1_A\mid {\mathcal{F}}_{\infty }]$ almost everywhere and in $L^1$. Each of these probabilities is positive almost everywhere on $A$: for example, integrating $1_A$ over the ${\mathcal{F}}_{\infty }$ measurable set $\{p_{\infty }^A=0\}$ gives zero. Countability of $\xi$ lets us choose a common full-measure set for every atom.

The conditional information functionstherefore converge almost everywhere to $f_{\infty }=I_{\mu }(\xi \mid {\mathcal{F}}_{\infty })$. We need an integrable bound on this sequence; convergence of the probabilities alone would not supply one after taking logarithms. The allowed maximal inequality for conditional information functions gives, for $F={\sup }_{k\ge 0}{f}_k$,Combining this with the bound by $\mu (A)$ and the tail integral formula for moments yieldsThus $0\le f_{\infty }\le F$, and the dominated convergence theorem gives $f_k\to f_{\infty }$ in $L^1$. In particular,because $H_n/n$ is the average of the decreasing sequence $(c_k)$.

The information chain rule, applied from the final observation backwards, gives the exact identityIndeed, the symbol at time $j$ is conditioned on times $j+1,\dots ,n-1$; pulling that conditional probability back by $T^j$ gives $f_{n-1-j}\circ T^j$. Measure preservation is sufficient for this pullback identity.

We now prove the required triangular ergodic averaging lemma in this instance. Put $g_M={\sup }_{k\ge M}|f_k-f_{\infty }|$. Then $g_M\downarrow 0$ almost everywhere, $g_M\le 2F$, and $\Vert g_M{\Vert }_1\to 0$. Split the difference between the displayed triangular sum and ${\sum }_{j=0}^{n-1}{f}_{\infty }(T^{j}x)$. Terms whose index is at least $M$ are bounded by ${\sum }_{j=0}^{n-1}{g}_M(T^{j}x)$. The remaining terms are at mostAfter division by $n$, each of these finitely many end terms tends to zero almost everywhere by the linear growth bound for integrable observables proved in Question 1. Applying the Birkhoff ergodic theorem to $g_M$, for every fixed $M$, therefore givesThese conditional expectations decrease to zero almost everywhere: they decrease, and their integrals $\int g_{M}d\mu$ tend to zero. Letting $M\to \infty$, then applying the Birkhoff ergodic theorem to $f_{\infty }$, proves

For $L^1$ convergence, measure preservation gives the direct estimateCombine this with $L^1$ convergence of the ergodic averages of $f_{\infty }$. Finally, $\int h_{\xi }d\mu =\int f_{\infty }d\mu =h_{\mu }(T,\xi )$. In the ergodic case the invariant sigma-algebra is trivial, completing the familiar form of the theorem:

The Birkhoff ergodic theorem states that if $T$ is a measure-preserving transformation and $f\in L^1(\mu )$, thenwhere $\mathcal{I}$ is the invariant sigma-algebra. The limit has the same integral as $f$. If $T$ is an ergodic transformation, the limit is the constant ${\int }_{X}fd\mu$ almost everywhere.

Let $T$ be a measure-preserving transformation of $(X,\mathcal{F},\mu )$ and let $f\in L^1(\mu )$. The Birkhoff ergodic theorem states thatalmost everywhere, where $\mathcal{I}$ is the invariant sigma-algebra. The limit is $T$-invariant and

Assume now that $\mu (X)=1$. For $M>0$, letbe the bounded truncation of $f$, and let $f_M^{\ast }$ be its Birkhoff limit. Since $|A_n{f}_M-f_M^{\ast }|\le 2M$ on a finite measure space, the dominated convergence theorem givesMeasure preservation and the triangle inequality give the $L^1$ contractionMoreover, the almost-everywhere convergence and Fatou lemma implyConsequentlySince $f_M\to f$ in $L^1$, letting $M\to \infty$ proves the L1 convergence in the Birkhoff ergodic theorem on a finite measure space:

Let $T(x)=x+\alpha (\mathrm{mod}1)$ with irrational $\alpha$. This irrational rotation of the circle preserves Lebesgue measure and is ergodic. Apply the Birkhoff ergodic theorem to the indicator function $1_A$. Its limitis invariant, hence almost everywhere constant by ergodicity. Its integral must equal that of $1_A$, so the constant is $\mu (A)$. Thereforefor Lebesgue measure-almost every $x\in (0,1]$.

Suppose $f_k\to f$ almost everywhere and ${\sup }_k|f_k|\in L^1$ in a probability measure-preserving system. Thenalmost everywhere and in $L^1$, where $\mathcal{I}$ is the invariant sigma-algebra. For the almost-everywhere assertion, bound the terms with index at least $M$ by ${\sup }_{k\ge M}|f_k-f|$, apply the Birkhoff ergodic theorem, and then let $M\to \infty$. The finitely many remaining end terms vanish by the linear growth bound for integrable observables.