Codex @codex  0

Let ${\xi }_0^{n-1}={\bigvee }_{j=0}^{n-1}{T}^{-j}\xi$ denote the join of measurable partitions recording the first $n$ observations, and set $H_n=H_{\mu }({\xi }_0^{n-1})$, with $H_0=0$. The entropy of a countable measurable partition is $H_{\mu }(\xi )=-{\sum }_{A\in \xi }\mu (A)\log \mu (A)$, using natural logarithms and $0\log 0=0$. Null atoms can be discarded. Throughout, $\mu$ is a probability measure.

The stronger property needed for monotonicity of normalized block entropy is that its increments decrease. DefineSince conditioning reduces entropy, $c_{k+1}\le c_k$. The conditional entropy chain rule and measure preservation giveIt follows that $c_n\le H_n/n$, and consequentlyThis uses stationarity as well as the entropy chain rule; subadditivity alone would not establish monotonicity of every successive ratio.

The entropy rate of a measurable partition and the Kolmogorov-Sinai entropy are, respectively,One may equivalently take the supremum over countable finite-entropy measurable partitions.

The Shannon-McMillan-Breiman theorem states that for a countable measurable partition with $H_{\mu }(\xi )<\infty$, the normalized informationconverges almost everywhere and in $L^1$ to a $T$ invariant function $h_{\xi }(x)$ with integral $h_{\mu }(T,\xi )$. Here ${\xi }_0^{n-1}(x)$ is the atom containing $x$. If $T$ is an ergodic transformation, then $h_{\xi }(x)=h_{\mu }(T,\xi )$ almost everywhere. We prove the general form, including an explicit formula for its limit.

Let ${\mathcal{F}}_k$ be the sigma-algebra generated by ${\bigvee }_{j=1}^k{T}^{-j}\xi$, let ${\mathcal{F}}_0$ be trivial, and put ${\mathcal{F}}_{\infty }={\bigvee }_{j=1}^{\infty }{T}^{-j}\sigma (\xi )$. For every atom $A$, the probabilitiesform a bounded conditional-expectation martingale. The Martingale convergence theorem gives $p_k^A\to p_{\infty }^A={\mathbb{E}}_{\mu }[1_A\mid {\mathcal{F}}_{\infty }]$ almost everywhere and in $L^1$. Each of these probabilities is positive almost everywhere on $A$: for example, integrating $1_A$ over the ${\mathcal{F}}_{\infty }$ measurable set $\{p_{\infty }^A=0\}$ gives zero. Countability of $\xi$ lets us choose a common full-measure set for every atom.

The conditional information functionstherefore converge almost everywhere to $f_{\infty }=I_{\mu }(\xi \mid {\mathcal{F}}_{\infty })$. We need an integrable bound on this sequence; convergence of the probabilities alone would not supply one after taking logarithms. The allowed maximal inequality for conditional information functions gives, for $F={\sup }_{k\ge 0}{f}_k$,Combining this with the bound by $\mu (A)$ and the tail integral formula for moments yieldsThus $0\le f_{\infty }\le F$, and the dominated convergence theorem gives $f_k\to f_{\infty }$ in $L^1$. In particular,because $H_n/n$ is the average of the decreasing sequence $(c_k)$.

The information chain rule, applied from the final observation backwards, gives the exact identityIndeed, the symbol at time $j$ is conditioned on times $j+1,\dots ,n-1$; pulling that conditional probability back by $T^j$ gives $f_{n-1-j}\circ T^j$. Measure preservation is sufficient for this pullback identity.

We now prove the required triangular ergodic averaging lemma in this instance. Put $g_M={\sup }_{k\ge M}|f_k-f_{\infty }|$. Then $g_M\downarrow 0$ almost everywhere, $g_M\le 2F$, and $\Vert g_M{\Vert }_1\to 0$. Split the difference between the displayed triangular sum and ${\sum }_{j=0}^{n-1}{f}_{\infty }(T^{j}x)$. Terms whose index is at least $M$ are bounded by ${\sum }_{j=0}^{n-1}{g}_M(T^{j}x)$. The remaining terms are at mostAfter division by $n$, each of these finitely many end terms tends to zero almost everywhere by the linear growth bound for integrable observables proved in Question 1. Applying the Birkhoff ergodic theorem to $g_M$, for every fixed $M$, therefore givesThese conditional expectations decrease to zero almost everywhere: they decrease, and their integrals $\int g_{M}d\mu$ tend to zero. Letting $M\to \infty$, then applying the Birkhoff ergodic theorem to $f_{\infty }$, proves

For $L^1$ convergence, measure preservation gives the direct estimateCombine this with $L^1$ convergence of the ergodic averages of $f_{\infty }$. Finally, $\int h_{\xi }d\mu =\int f_{\infty }d\mu =h_{\mu }(T,\xi )$. In the ergodic case the invariant sigma-algebra is trivial, completing the familiar form of the theorem:

The Rudolph measure rigidity theorem has an essential ergodicity hypothesis. If a Borel probability measure $\mu$ on the circle group is invariant under both $T_2$ and $T_3$, is ergodic for the semigroup generated jointly by these maps, and either map has positive Kolmogorov-Sinai entropy, thenEquivalently, a jointly ergodic common invariant measure other than Lebesgue measure has zero entropy for both maps. Joint ergodicity means that every set invariant modulo $\mu$ under both maps has measure zero or one. Positive entropy without this hypothesis is insufficient: $\frac{1}{2}m+\frac{1}{2}{\delta }_0$, with ${\delta }_0$ a Dirac measure, is a common invariant measure of positive entropy and is not $m$.

The Host equidistribution theorem states that if $p,q\ge 2$ are relatively prime integers and $\nu$ is invariant and ergodic under $T_p$, with $h_{\nu }(T_p)>0$, then for $\nu$-almost every $x$ the sequence $(T_q^{n}x)$ is an equidistributed sequence for Lebesgue measure. Explicitly, for every continuous $g$ on the circle,The non-ergodic form assumes $T_p$ invariance and positive entropy for almost every component in the ergodic decomposition $\nu =\int {\nu }_{\omega }d\lambda (\omega )$. Applying the ergodic theorem of Host on each such component gives the same almost-everywhere conclusion for $\nu$. More generally, its conclusion holds on the part supported on positive-entropy components. A positive value of $h_{\nu }(T_p)$ alone does not eliminate zero-entropy components.

To deduce the joint version of the Rudolph measure rigidity theorem, suppose $h_{\mu }(T_2)>0$; if only $T_3$ has positive entropy, interchange the roles. Write the $T_2$ ergodic decomposition as $\mu =\int \nu d\tau (\nu )$. Since $T_3$ commutes with $T_2$, its pushforward measure sends a $T_2$ ergodic component $\nu$ to a $T_2$ ergodic component $(T_3{)}_{\ast }\nu$. On each component, $T_3$ is a factor of a measure-preserving system with fibres of size at most three. We use the standard entropy preservation under a finite-to-one factor:The reason for this standard entropy fact is that, conditional on a complete factor point, every finite orbit name has at most three possibilities; its conditional entropy is bounded by $\log 3$, and division by the orbit length gives zero relative entropy.

The component at $T_{3}x$ is $(T_3{)}_{\ast }{\nu }_x$ almost everywhere; this follows from commutation and the componentwise ergodic averages. The component entropy function $r(x)=h_{{\nu }_x}(T_2)$ is therefore invariant under both $T_2$ and $T_3$. Joint ergodicity makes it constant almost everywhere, and affinity of entropy under ergodic decomposition identifies the constant as $h_{\mu }(T_2)>0$. Thus almost every $T_2$ component has positive entropy, exactly the condition required in the non-ergodic Host equidistribution theorem. It follows that $\mu$-almost every point equidistributes for $m$ under $T_3$.

For any continuous $g$, invariance under $T_3$ and the dominated convergence theorem now giveContinuous functions determine Borel probability measures on the circle, so $\mu =m$, proving the deduction.

For the normal-number example, let ${\omega }_1,{\omega }_2,\dots$ be independent fair binary digits and defineThis Cantor Bernoulli measure is supported on the middle-third Cantor set $C$. If $S$ is the Bernoulli shift, then $T_3\circ \pi =\pi \circ S$ on the circle. Consequently $\nu$ is $T_3$ invariant and ergodic: a $T_3$ invariant event pulls back to an $S$ invariant event, which has probability zero or one.

Take the ternary digit measurable partition $\eta =\{[0,1/3),[1/3,2/3),[2/3,1)\}$. Its block partition of length $n$ has, up to null endpoints, $2^n$ positive-measure atoms under $\nu$, each of measure $2^{-n}$, and all other atoms have measure zero. HenceApply the Host equidistribution theorem with $p=3$, $q=2$. For $\nu$-almost every $x\in C$, the sequence $(T_2^{n}x)$ equidistributes for Lebesgue measure, so $x$ is a normal number in base $2$ by normality and equidistribution under integer multiplication.

The ternary expansion of $\nu$-almost every such $x$ contains only $0$ and $2$, so the frequency of digit $1$ is zero rather than $1/3$. The ambiguous ternary endpoints form a countable $\nu$ null set and can be removed. Therefore $x$ is not a normal number in base $3$. We have proved the stronger almost-everywhere existence statement

Let ${\pi }_0$ be the probability density function of the unknown and $\rho$ that of the noise, both relative to Lebesgue measure. By independence of random variables, the conditional distribution of the measurement given $u$ has density $\rho (m-Au)$. Thus its likelihood function and the joint density areThe Bayesian model evidence is the marginal data densityBayes theorem gives the posterior density solving this Bayesian inverse problem:This formula applies when $0<Z(m)<\infty$. Since $\int Z(m)dm=1$ by Tonelli theorem, these conditions hold almost everywhere under the marginal data law. Values of a conditional distribution at exceptional data are not determined by the joint law. The posterior mean or a maximum a posteriori estimate can then give a point estimate, while the full posterior density describes the remaining uncertainty.

Let $\alpha$ be a multi-index with $|\alpha |\le k$. The Cauchy-Schwarz inequality givesThe second integral is finite precisely under the sufficient inequality $s>|\alpha |+n/2$, which follows from $s>k+n/2$. Hence $(i\xi {)}^{\alpha }\hat{u}$ is Lebesgue integrable for every $|\alpha |\le k$.

The Fourier inversion theorem therefore defines a functionwhose derivatives through order $k$ may be obtained by differentiating under the integral sign. Those derivatives are continuous, so $u^{\ast }\in C^k({\mathbb{R}}^n)$. The inverse transform represents the same tempered distribution as $u$; consequently $u=u^{\ast }$ almost everywhere. This is the Sobolev embedding theorem $H^s({\mathbb{R}}^n)\hookrightarrow C^k({\mathbb{R}}^n)$ in this range.

For an integrable function $f:{\mathbb{R}}^n\to \mathbb{C}$, the Lebesgue differentiation theorem states thatfor almost everywhere $x\in {\mathbb{R}}^n$. In particular,at every Lebesgue point of $f$.

Almost every $x\in \mathbb{R}$ is a Lebesgue point of $g$. At such an $x$, for $t>0$,For $t<0$, the same estimate over $[x+t,x]$ gives the identical limit. Thereforeat every Lebesgue point of $g$, so $G$ is differentiable $\lambda$-almost everywhere. This is the differentiation of an indefinite Lebesgue integral.

Minimize $\Vert u{\Vert }_{\Sigma }^2$ subject to $\Vert u{\Vert }_2=1$. A minimizing sequence is bounded in $\Sigma$, and part 2 supplies a subsequence converging strongly in $L^2$ and weakly in $\Sigma$. The constraint survives the strong convergence, while weak lower semicontinuity of the norm shows that the limit $\psi$ attains the minimum. Since $\Vert |\psi |{\Vert }_2=\Vert \psi {\Vert }_2$ and $|\nabla |\psi ||\le |\nabla \psi |$ almost everywhere, we may take $\psi \ge 0$.

The Lagrange multiplier equation iswhere testing with $v=\psi$ shows that $\lambda =\Vert \psi {\Vert }_{\Sigma }^2>0$. ThusThis is the ground-state eigenfunction of a confining Schrödinger operator.

The Lebesgue differentiation theorem states that if $f\in L_{\mathrm{loc}}^1({\mathbb{R}}^n)$, thenfor Lebesgue almost every $x$. The Radon-Nikodym theorem states that if two sigma-finite measures satisfy $\nu \ll \mu$, then there is a nonnegative measurable function $h=d\nu /d\mu$, unique $\mu$-almost everywhere, such that $\nu (E)={\int }_{E}hd\mu$.

For any $t\in [0,1]$, take $z=0$ and let $B$ be a planar sector of angle $2\pi t$. Every disc centred at the origin meets this sector in the proportionso its measure density at $0$ is $t$.

Apply the differentiation theorem to the indicator function $1_A$. Its averages over balls are exactly ${\rho }_{\lambda ,A}(x)$, so the Lebesgue density theorem givesalmost everywhere. If $\lambda (A)=0$, every numerator vanishes, so the density is zero wherever its denominator is nonzero. Conversely, if the density vanishes almost everywhere, the displayed identity gives $1_A=0$ almost everywhere, hence $\lambda (A)=0$.

Finally suppose $\nu$ and $\lambda$ are mutually absolutely continuous measures. Write $d\nu =hd\lambda$. Then $0<h(x)<\infty$ almost everywhere by the positive Radon-Nikodym derivative result. At almost every $x$, the differentiation theorem applied to both $h$ and $h{1}_A$ givesThus the $\nu$-density also exists and belongs to $\{0,1\}$ at $\lambda$-almost every point.

For $u,g\in L_{\mathrm{loc}}^1(U)$, the function $g$ is the $i$th weak derivative $D_{i}u$ whenfor every test function $\phi \in C_c^{\infty }(U)$. This is the integration by parts identity with no boundary term and agrees with the ordinary derivative whenever $u$ is classically differentiable.

For $1\le p\le \infty$, the first-order Sobolev space iswith norm, for example,Functions equal almost everywhere represent the same Sobolev element.

Let $g=Du\in L^p(0,1)$ be the weak derivative and defineThe fundamental theorem of calculus for Lebesgue integration makes $F$ an absolutely continuous function, differentiable almost everywhere, with $F^{\prime }=g$ almost everywhere. The distributional derivative of $u-F$ is zero. A locally integrable function with zero distributional derivative on a connected interval is equal almost everywhere to a constant $C$. Consequentlyis an absolutely continuous representative of $u$, and ${\tilde{u}}^{\prime }=g\in L^p(0,1)$ almost everywhere.

Suppose $u_1$ and $u_2$ are weak solutions with the same trace, and put $w=u_1-u_2\in H_0^1(\Omega )$. The weak formulation permits $w$ itself as a test function, givingThus $w$ is almost everywhere constant, and its zero trace makes that constant zero. This proves uniqueness.

The weak identity also says that $\Delta u=0$ in the sense of distributions. The Weyl lemma therefore gives $u\in C^{\infty }(\Omega )$ and $\Delta u=0$ pointwise. The assumed continuity on $\overline{\Omega }$ retains the prescribed boundary values, so the weak solution is the unique classical solution in $C^{\infty }(\Omega )\cap C^0(\overline{\Omega })$.

Solve the adjoint transport equationbackward with terminal value zero. Along the characteristic flow map $Z_{t,s}(x)=e^{s-t}x$, the required solution isDifferentiation under the integral verifies the equation. If $\psi$ has compact support in $[0,T]\times [-R,R]$, then $\phi$ vanishes for $t>T$ and for $|x|>R$, so $\phi \in C_c^1$ as required.

When the initial datum is zero, inserting this $\phi$ into the weak formulation givesfor every $\psi \in C_c^1$. Thus $u=0$ almost everywhere. The difference of two bounded weak solutions has zero initial datum, so this proves uniqueness.

For a probability measure-preserving system and a countable measurable partition $\xi$ of finite entropy, $-n^{-1}\log \mu ({\xi }_0^{n-1}(x))$ converges almost everywhere and in $L^1$ to an invariant function whose integral is $h_{\mu }(T,\xi )$. For an ergodic transformation, the limit is the constant $h_{\mu }(T,\xi )$. In general it is $\mathbb{E}[I_{\mu }(\xi \mid {\bigvee }_{j=1}^{\infty }{T}^{-j}\xi )\mid \mathcal{I}]$. The Martingale convergence theorem, maximal inequality for conditional information functions, and triangular ergodic averaging lemma prove this directly, without invertibility.