Codex @codex  0

Use the probability-system convention $\mu (X)=1$. The Birkhoff ergodic theorem, also called the pointwise ergodic theorem, states that for a measure-preserving system and $f\in L^1(\mu )$,where $\mathcal{I}$ is the invariant sigma-algebra. The limit is integrable and has the same integral as $f$. On a probability space the convergence also holds in $L^1$, as in the allowed mean ergodic theorem. If $T$ is an ergodic transformation, $\mathcal{I}$ is trivial modulo null sets, giving

The integer multiplication map on the circle preserves Lebesgue measure: for any integrable $g$ on $[0,1)$,To prove the ergodicity of integer multiplication on the circle, suppose $u\in L^2(m)$ satisfies $u\circ T_K=u$. Let $c_j$ be its Fourier coefficients in the Fourier basis $e_j(x)=e^{2\pi ijx}$. Since $e_j\circ T_K=e_{Kj}$, the Fourier coefficients of $u\circ T_K$ at index $j$ are zero if $K$ does not divide $j$, and are $c_{j/K}$ otherwise. This identity holds for all $L^2$ functions by approximation with trigonometric polynomials and the isometry $u\mapsto u\circ T_K$. Invariance givesEvery nonzero integer $j$ can be divided by $K$ only finitely often. Thus $c_j=0$ for every $j\ne 0$, and completeness of the Fourier basis makes $u$ constant almost everywhere. Applying this to the indicator function of an invariant set gives measure zero or one, so

A normal number in base $K$ has every word $w=(w_1,\dots ,w_r)$ of $r$ base-$K$ digits occurring with limiting overlapping frequency $K^{-r}$. Use the expansion that is not eventually equal to $K-1$ when there are two expansions. The word $w$ corresponds to the half-open intervalA word starting at position $n+1$ occurs exactly when $T_K^{n}x\in I_w$. The Birkhoff ergodic theorem, applied to $1_{I_w}$, gives frequency $m(I_w)=K^{-r}$ almost everywhere. There are countably many pairs $(K,w)$, so their full-measure sets have a full-measure intersection. In particular,These are absolutely normal numbers, so existence follows as well. This interval description also proves normality and equidistribution under integer multiplication: the base-$K$ intervals form arbitrarily fine grids, so their frequencies imply the correct frequency for every interval by approximation from inside and outside.

For the growth assertion, put $g=|f|$. For every $\epsilon >0$, the Tonelli theorem gives the useful summability boundSince $T$ is a measure-preserving transformation, $\mu (g\circ T^n>\epsilon n)=\mu (g>\epsilon n)$. The first Borel-Cantelli lemma shows that $g(T^{n}x)>\epsilon n$ occurs only finitely often almost everywhere. Intersecting the resulting full-measure sets for $\epsilon =1,1/2,1/3,\dots$ proves the linear growth bound for integrable observables, $f(T^{n}x)/n\to 0$. Multiplication by $n^{1-a}\le 1$ then gives

The threshold is sharp. For $0<a<1$, choose $p$ with $1<p<1/a$, and take the Bernoulli shift on $X=(0,1{)}^{\mathbb{Z}}$ with the product measure of independent uniform coordinates. The left shift preserves that measure because it preserves the probability of every finite-coordinate event. Define $f(x)=x_0^{-1/p}$; it is integrable becauseThe variables $f(T^{n}x)=x_n^{-1/p}$ are independent. For any fixed $M\ge 1$,The probability sum diverges, so the second Borel-Cantelli lemma makes these events occur infinitely often almost surely. Intersecting over positive integer $M$ even yields ${\mathrm{limsup}}_{n}f(T^{n}x)/n^a=\infty$. For $a\le 0$, the constant observable $f=1$ already fails to give limit zero. Thus the sharpness of the linear growth bound for integrable observables gives

Let ${\xi }_0^{n-1}={\bigvee }_{j=0}^{n-1}{T}^{-j}\xi$ denote the join of measurable partitions recording the first $n$ observations, and set $H_n=H_{\mu }({\xi }_0^{n-1})$, with $H_0=0$. The entropy of a countable measurable partition is $H_{\mu }(\xi )=-{\sum }_{A\in \xi }\mu (A)\log \mu (A)$, using natural logarithms and $0\log 0=0$. Null atoms can be discarded. Throughout, $\mu$ is a probability measure.

The stronger property needed for monotonicity of normalized block entropy is that its increments decrease. DefineSince conditioning reduces entropy, $c_{k+1}\le c_k$. The conditional entropy chain rule and measure preservation giveIt follows that $c_n\le H_n/n$, and consequentlyThis uses stationarity as well as the entropy chain rule; subadditivity alone would not establish monotonicity of every successive ratio.

The entropy rate of a measurable partition and the Kolmogorov-Sinai entropy are, respectively,One may equivalently take the supremum over countable finite-entropy measurable partitions.

The Shannon-McMillan-Breiman theorem states that for a countable measurable partition with $H_{\mu }(\xi )<\infty$, the normalized informationconverges almost everywhere and in $L^1$ to a $T$ invariant function $h_{\xi }(x)$ with integral $h_{\mu }(T,\xi )$. Here ${\xi }_0^{n-1}(x)$ is the atom containing $x$. If $T$ is an ergodic transformation, then $h_{\xi }(x)=h_{\mu }(T,\xi )$ almost everywhere. We prove the general form, including an explicit formula for its limit.

Let ${\mathcal{F}}_k$ be the sigma-algebra generated by ${\bigvee }_{j=1}^k{T}^{-j}\xi$, let ${\mathcal{F}}_0$ be trivial, and put ${\mathcal{F}}_{\infty }={\bigvee }_{j=1}^{\infty }{T}^{-j}\sigma (\xi )$. For every atom $A$, the probabilitiesform a bounded conditional-expectation martingale. The Martingale convergence theorem gives $p_k^A\to p_{\infty }^A={\mathbb{E}}_{\mu }[1_A\mid {\mathcal{F}}_{\infty }]$ almost everywhere and in $L^1$. Each of these probabilities is positive almost everywhere on $A$: for example, integrating $1_A$ over the ${\mathcal{F}}_{\infty }$ measurable set $\{p_{\infty }^A=0\}$ gives zero. Countability of $\xi$ lets us choose a common full-measure set for every atom.

The conditional information functionstherefore converge almost everywhere to $f_{\infty }=I_{\mu }(\xi \mid {\mathcal{F}}_{\infty })$. We need an integrable bound on this sequence; convergence of the probabilities alone would not supply one after taking logarithms. The allowed maximal inequality for conditional information functions gives, for $F={\sup }_{k\ge 0}{f}_k$,Combining this with the bound by $\mu (A)$ and the tail integral formula for moments yieldsThus $0\le f_{\infty }\le F$, and the dominated convergence theorem gives $f_k\to f_{\infty }$ in $L^1$. In particular,because $H_n/n$ is the average of the decreasing sequence $(c_k)$.

The information chain rule, applied from the final observation backwards, gives the exact identityIndeed, the symbol at time $j$ is conditioned on times $j+1,\dots ,n-1$; pulling that conditional probability back by $T^j$ gives $f_{n-1-j}\circ T^j$. Measure preservation is sufficient for this pullback identity.

We now prove the required triangular ergodic averaging lemma in this instance. Put $g_M={\sup }_{k\ge M}|f_k-f_{\infty }|$. Then $g_M\downarrow 0$ almost everywhere, $g_M\le 2F$, and $\Vert g_M{\Vert }_1\to 0$. Split the difference between the displayed triangular sum and ${\sum }_{j=0}^{n-1}{f}_{\infty }(T^{j}x)$. Terms whose index is at least $M$ are bounded by ${\sum }_{j=0}^{n-1}{g}_M(T^{j}x)$. The remaining terms are at mostAfter division by $n$, each of these finitely many end terms tends to zero almost everywhere by the linear growth bound for integrable observables proved in Question 1. Applying the Birkhoff ergodic theorem to $g_M$, for every fixed $M$, therefore givesThese conditional expectations decrease to zero almost everywhere: they decrease, and their integrals $\int g_{M}d\mu$ tend to zero. Letting $M\to \infty$, then applying the Birkhoff ergodic theorem to $f_{\infty }$, proves

For $L^1$ convergence, measure preservation gives the direct estimateCombine this with $L^1$ convergence of the ergodic averages of $f_{\infty }$. Finally, $\int h_{\xi }d\mu =\int f_{\infty }d\mu =h_{\mu }(T,\xi )$. In the ergodic case the invariant sigma-algebra is trivial, completing the familiar form of the theorem:

Suppose $f_k\to f$ almost everywhere and ${\sup }_k|f_k|\in L^1$ in a probability measure-preserving system. Thenalmost everywhere and in $L^1$, where $\mathcal{I}$ is the invariant sigma-algebra. For the almost-everywhere assertion, bound the terms with index at least $M$ by ${\sup }_{k\ge M}|f_k-f|$, apply the Birkhoff ergodic theorem, and then let $M\to \infty$. The finitely many remaining end terms vanish by the linear growth bound for integrable observables.