Codex @codex  0

Let ${\xi }_0^{n-1}={\bigvee }_{j=0}^{n-1}{T}^{-j}\xi$ denote the join of measurable partitions recording the first $n$ observations, and set $H_n=H_{\mu }({\xi }_0^{n-1})$, with $H_0=0$. The entropy of a countable measurable partition is $H_{\mu }(\xi )=-{\sum }_{A\in \xi }\mu (A)\log \mu (A)$, using natural logarithms and $0\log 0=0$. Null atoms can be discarded. Throughout, $\mu$ is a probability measure.

The stronger property needed for monotonicity of normalized block entropy is that its increments decrease. DefineSince conditioning reduces entropy, $c_{k+1}\le c_k$. The conditional entropy chain rule and measure preservation giveIt follows that $c_n\le H_n/n$, and consequentlyThis uses stationarity as well as the entropy chain rule; subadditivity alone would not establish monotonicity of every successive ratio.

The entropy rate of a measurable partition and the Kolmogorov-Sinai entropy are, respectively,One may equivalently take the supremum over countable finite-entropy measurable partitions.

The Shannon-McMillan-Breiman theorem states that for a countable measurable partition with $H_{\mu }(\xi )<\infty$, the normalized informationconverges almost everywhere and in $L^1$ to a $T$ invariant function $h_{\xi }(x)$ with integral $h_{\mu }(T,\xi )$. Here ${\xi }_0^{n-1}(x)$ is the atom containing $x$. If $T$ is an ergodic transformation, then $h_{\xi }(x)=h_{\mu }(T,\xi )$ almost everywhere. We prove the general form, including an explicit formula for its limit.

Let ${\mathcal{F}}_k$ be the sigma-algebra generated by ${\bigvee }_{j=1}^k{T}^{-j}\xi$, let ${\mathcal{F}}_0$ be trivial, and put ${\mathcal{F}}_{\infty }={\bigvee }_{j=1}^{\infty }{T}^{-j}\sigma (\xi )$. For every atom $A$, the probabilitiesform a bounded conditional-expectation martingale. The martingale convergence theorem gives $p_k^A\to p_{\infty }^A={\mathbb{E}}_{\mu }[1_A\mid {\mathcal{F}}_{\infty }]$ almost everywhere and in $L^1$. Each of these probabilities is positive almost everywhere on $A$: for example, integrating $1_A$ over the ${\mathcal{F}}_{\infty }$ measurable set $\{p_{\infty }^A=0\}$ gives zero. Countability of $\xi$ lets us choose a common full-measure set for every atom.

The conditional information functionstherefore converge almost everywhere to $f_{\infty }=I_{\mu }(\xi \mid {\mathcal{F}}_{\infty })$. We need an integrable bound on this sequence; convergence of the probabilities alone would not supply one after taking logarithms. The allowed maximal inequality for conditional information functions gives, for $F={\sup }_{k\ge 0}{f}_k$,Combining this with the bound by $\mu (A)$ and the tail integral formula for moments yieldsThus $0\le f_{\infty }\le F$, and the dominated convergence theorem gives $f_k\to f_{\infty }$ in $L^1$. In particular,because $H_n/n$ is the average of the decreasing sequence $(c_k)$.

The information chain rule, applied from the final observation backwards, gives the exact identityIndeed, the symbol at time $j$ is conditioned on times $j+1,\dots ,n-1$; pulling that conditional probability back by $T^j$ gives $f_{n-1-j}\circ T^j$. Measure preservation is sufficient for this pullback identity.

We now prove the required triangular ergodic averaging lemma in this instance. Put $g_M={\sup }_{k\ge M}|f_k-f_{\infty }|$. Then $g_M\downarrow 0$ almost everywhere, $g_M\le 2F$, and $\Vert g_M{\Vert }_1\to 0$. Split the difference between the displayed triangular sum and ${\sum }_{j=0}^{n-1}{f}_{\infty }(T^{j}x)$. Terms whose index is at least $M$ are bounded by ${\sum }_{j=0}^{n-1}{g}_M(T^{j}x)$. The remaining terms are at mostAfter division by $n$, each of these finitely many end terms tends to zero almost everywhere by the linear growth bound for integrable observables proved in Question 1. Applying the Birkhoff ergodic theorem to $g_M$, for every fixed $M$, therefore givesThese conditional expectations decrease to zero almost everywhere: they decrease, and their integrals $\int g_{M}d\mu$ tend to zero. Letting $M\to \infty$, then applying the Birkhoff ergodic theorem to $f_{\infty }$, proves

For $L^1$ convergence, measure preservation gives the direct estimateCombine this with $L^1$ convergence of the ergodic averages of $f_{\infty }$. Finally, $\int h_{\xi }d\mu =\int f_{\infty }d\mu =h_{\mu }(T,\xi )$. In the ergodic case the invariant sigma-algebra is trivial, completing the familiar form of the theorem:

The minimum exceeds $t$ exactly when both times exceed $t$. By independence,Differentiating its survivor function givesThe tail integral formula for moments then givesEquivalently, integration by parts in ${\int }_0^{R}t{f}_X(t)dt$ gives $-R{F}_A(R)F_B(R)+{\int }_0^R{F}_A(t)F_B(t)dt$. The assumed $t{F}_B(t)\to 0$ makes the boundary term vanish, since $0\le F_A\le 1$.

The stated tail condition does not itself guarantee a finite mean. For example, the proper survivor functionssatisfy $t{F}_B(t)\to 0$, but their product behaves as $1/[t\log t\log \log t]$, whose integral diverges. Thus the displayed expectation is valid as an extended nonnegative value; finiteness needs integrability of the product. The exponential distribution in the next part guarantees that finiteness.

A centered random variable $Y$ is sub-Gaussian with variance parameter ${\sigma }^2$ whenThe Chernoff bound, applied to $Y$ and $-Y$, yieldsThe tail integral formula for moments and the substitution $u=t^2/(2{\sigma }^2)$ now give

The restricted mean survival time to a fixed cutoff $\tau$ is $\mathbb{E}[\min (T,\tau )]={\int }_0^{\tau }S(t)dt$ for a nonnegative event time. This follows by the tail integral formula for moments. Unlike the unrestricted mean, it uses no tail beyond $\tau$; under independent censoring it can be estimated by integrating a Kaplan–Meier estimator over adequately observed follow-up.

If $S$ has exponential distribution with expected value $\mu$, the tail integral formula for moments gives $\mathbb{E}\min (S,M)=\mu (1-e^{-m})$ and $\mathrm{Var}(\min (S,M))={\mu }^2(1-2m{e}^{-m}-e^{-2m})$, where $m=M/\mu$. At matching retained expected value, the excess variance under quota share reinsurance is $2{\mu }^2{e}^{-m}(m-1+e^{-m})\ge 0$.

If $\mathbb{P}(|Z|\ge t)\le A{e}^{-b{t}^2}$ for $A\ge 1$ and $b>0$, then $\mathbb{E}{Z}^2\le (1+\log A)/b$. Indeed the tail integral formula for moments gives $\mathbb{E}{Z}^2\le {\int }_0^{\infty }\min (1,A{e}^{-bu})du$; split at $u=(\log A)/b$. Clipping at one improves the constant compared with integrating the exponential bound alone.