Entropy rate of a measurable partition ($h_{\mu }(T,\xi )$)

The entropy rate of $\xi$ isThe numerator is a subadditive sequence, so the limit exists and equals the infimum of the displayed ratios.

For a finite measurable partition $\xi$, put ${\mathcal{F}}_1=\sigma ({\bigvee }_{j\ge 1}{T}^{-j}\xi )$. Then $h_{\mu }(T,\xi )=H_{\mu }(\xi \mid {\mathcal{F}}_1)={\lim }_{n\to \infty }{H}_{\mu }(\xi \mid {\xi }_1^n)$. The backward entropy chain rule writes $H_{\mu }({\xi }_0^{N-1})$ as the sum of the first $N$ decreasing finite-future conditional entropies. Their averages have the same limit. Increasing conditioning fields converge by the martingale convergence theorem.

For ${\mathcal{F}}_L=\sigma ({\bigvee }_{j\ge L}{T}^{-j}\xi )$ and a finite measurable partition $\xi$, $H_{\mu }({\xi }_0^{L-1}\mid {\mathcal{F}}_L)=L{h}_{\mu }(T,\xi )$. The backward conditional entropy chain rule expresses the left side as ${\sum }_{j=0}^{L-1}{H}_{\mu }(T^{-j}\xi \mid {\mathcal{F}}_{j+1})$. Each term equals $h_{\mu }(T,\xi )$ by measure preservation and the infinite-future formula for partition entropy rate. No invertibility is needed.

For a probability measure-preserving system and a countable measurable partition $\xi$ of finite entropy, $-n^{-1}\log \mu ({\xi }_0^{n-1}(x))$ converges almost everywhere and in $L^1$ to an invariant function whose integral is $h_{\mu }(T,\xi )$. For an ergodic transformation, the limit is the constant $h_{\mu }(T,\xi )$. In general it is $\mathbb{E}[I_{\mu }(\xi \mid {\bigvee }_{j=1}^{\infty }{T}^{-j}\xi )\mid \mathcal{I}]$. The martingale convergence theorem, maximal inequality for conditional information functions, and triangular ergodic averaging lemma prove this directly, without invertibility.

For a stationary process associated with a finite-entropy measurable partition, $H_{\mu }({\xi }_0^{n-1})/n$ is non-increasing. The increments $c_k=H_{\mu }(\xi \mid {\bigvee }_{j=1}^k{T}^{-j}\xi )$ are non-increasing by conditioning reduces entropy, and the chain rule gives $H_{\mu }({\xi }_0^{n-1})={\sum }_{k=0}^{n-1}{c}_k$. Averages of a decreasing sequence are decreasing.

The Kolmogorov-Sinai entropy is $h_{\mu }(T)={\sup }_{\xi }{h}_{\mu }(T,\xi )$ over finite measurable partitions.

A probability measure-preserving system has completely positive entropy if $h_{\mu }(T,\xi )>0$ for every finite measurable partition with $H_{\mu }(\xi )>0$. Equivalently, its Pinsker sigma-algebra is trivial. Since finite partitions measurable in a partition tail have zero entropy rate, this property forces every finite-partition tail sigma-algebra of a measurable partition to be trivial.

For an invertible measure-preserving system, a generating measurable partition $\xi$ has $\sigma ({\bigvee }_{j\in \mathbb{Z}}{T}^{-j}\xi )=\mathcal{B}$ modulo null sets. Thus knowing the entire two-sided itinerary determines all measurable observations. A finite or countable finite-entropy generating partition computes the system entropy through the Kolmogorov-Sinai generator theorem.

A one-sided generator is a measurable partition $\xi$ with $\sigma ({\bigvee }_{j\ge 0}{T}^{-j}\xi )=\mathcal{B}$ modulo null sets. Equivalently, every measurable set can be approximated in measure by unions of atoms of finite forward-name blocks. A one-sided coordinate partition generates a one-sided Bernoulli shift; for a nontrivial base distribution, it does not generate the two-sided version, whose negative coordinates are independent of the nonnegative ones.

If an invertible probability measure-preserving system has a finite one-sided generator $\xi$, then its future sigma-algebra ${\mathcal{F}}_1=T^{-1}\mathcal{B}$ equals $\mathcal{B}$ modulo null sets. Hence $H_{\mu }(\xi \mid {\mathcal{F}}_1)=0$, and the infinite-future formula for partition entropy rate and Kolmogorov-Sinai generator theorem give $h_{\mu }(T)=0$. Invertibility matters: a fair binary one-sided Bernoulli shift has entropy $\log 2$ and a finite one-sided generator.

For an invertible probability measure-preserving system with a finite or countable finite-entropy generating measurable partition $\xi$, $h_{\mu }(T)=h_{\mu }(T,\xi )$. For a noninvertible system the same conclusion holds for a finite-entropy one-sided generator. The finite-entropy hypothesis is part of this form of the theorem.

Let $p,q\ge 2$ be relatively prime integers. If $\nu$ is invariant and ergodic for the integer multiplication map on the circle $T_p$ and $h_{\nu }(T_p)>0$, then for $\nu$-almost every $x$ the sequence $(T_q^{n}x)$ is an equidistributed sequence for Lebesgue measure. For a non-ergodic $T_p$ invariant measure, the same conclusion holds if almost every measure in its ergodic decomposition has positive entropy. Positive entropy of the whole measure alone does not suffice.

A probability measure on the circle invariant under $T_2$ and $T_3$ and ergodic for their jointly generated semigroup is Lebesgue measure if either transformation has positive Kolmogorov-Sinai entropy. Without joint ergodicity, a mixture of Lebesgue measure and an atomic invariant measure shows why positive entropy alone is insufficient.

For standard probability systems, a factor of a measure-preserving system with at most $q$ points in every fibre preserves Kolmogorov-Sinai entropy. Conditional on the complete factor point, every finite orbit name has at most $q$ possibilities, so its conditional entropy is at most $\log q$. Dividing by the orbit length gives zero relative entropy.

The Pinsker sigma-algebra consists of the measurable sets $A$ whose binary partition $\{A,X\setminus A\}$ has entropy rate zero. It is a sigma-algebra and is the largest invariant factor with zero Kolmogorov-Sinai entropy.

A set belongs to the Pinsker sigma-algebra exactly when, modulo a null set, it belongs to $\mathcal{T}(\xi )$ for some finite measurable partition $\xi$. For a zero-entropy binary partition one may take that partition itself: zero conditional entropy makes its present atom measurable from every remote future.

For a measure-preserving action of $\mathbb{Z}$ and a finite partition $\xi$,The reverse inequality to the interval formula follows from Shearer's inequality applied to translates of $F$ along long intervals; the uncovered boundary has sublinear size.