Codex @codex  0

Harris' inequality states that, for a product measure on $\{0,1{\}}^n$ with independent Bernoulli random variables of parameters $p_i\in [0,1]$, increasing events $E,F$ satisfyIn particular, the uniform probability measure on the Boolean lattice gives $|E\cap F|\ge |E||F|/2^n$. We give a direct mathematical induction proof, so no stronger correlation theorem is assumed.

Prove more generally that $\mathbb{E}[fg]\ge \mathbb{E}[f]\mathbb{E}[g]$ for real coordinatewise increasing functions $f,g$ on the finite cube. For $n=0$ both are constant and equality holds. For the inductive step let $p=p_n$, and let $f_j,g_j$ be the restrictions obtained by setting the last coordinate equal to $j\in \{0,1\}$. Write $a_j=\mathbb{E}[f_j]$, $b_j=\mathbb{E}[g_j]$, using the product measure on the other coordinates. Monotonicity gives $a_0\le a_1$, $b_0\le b_1$. By the inductive hypothesis and conditional expectation,The nonnegative correction term also handles $p=0$ and $p=1$. Taking indicator functions $f=1_E$, $g=1_F$ proves Harris' inequality.

For later use, an increasing event $E$ and a decreasing event $G$ have negative correlation of increasing and decreasing events. The complement $G^c$ is an increasing event, so

Use the natural filtration ${\mathcal{F}}_k=\sigma (X_1,\dots ,X_k)$ and define $M_k=S_k-mk$. The increments $X_k-m$ are centered, independent, and square integrable, so their conditional expectations given the past are zero. Thus $(M_k)$ is a square-integrable martingale with $M_0=0$. Independence givesApplying the Doob L2 maximal inequality yieldsThis includes ${\sigma }^2=0$, when every centered increment vanishes almost surely.

For the converse in the characterization of a martingale by stopped expectations, fix $n\ge 0$ and $A\in {\mathcal{F}}_n$. Define the bounded stopping timeIndeed, its only nontrivial sublevel event is $\{T_A\le n\}=A^c\in {\mathcal{F}}_n$. Applying the assumed stopped-expectation equality to $T_A$ and to the deterministic stopping time $n$ givesThis holds for every $A\in {\mathcal{F}}_n$. Since $M_n$ is adapted and both variables are integrable, the defining test-event property of conditional expectation yieldsTogether with the given adaptation and integrability, this is exactly the martingale property.

For the forward implication, let $(M_n)$ be a martingale and let the stopping time $T$ satisfy $0\le T\le N$ for a deterministic integer $N$. The stopped variable is integrable because it uses only finitely many integrable values. The pathwise identityexpresses it through martingale increments. Since $\{T\ge j\}=\{T>j-1\}\in {\mathcal{F}}_{j-1}$, each summand has expectation zero by conditional expectation. ConsequentlyThis proves the bounded-time case of the optional stopping theorem directly; no assumptions about an unbounded stopping time are being used.

First suppose $|x+z|\le r$ for some lattice point $-z$. If $|x+z|=r$, the desired hit is already at time zero. If $|x+z|<r$, finite exit from that bounded open ball and continuity give the required sphere hit almost surely.

It remains to consider $|x+z|>r$ for every $z\in {\mathbb{Z}}^3$. At each integer time $n$, choose a nearest lattice point $-z_n$ to $B_n$ by rounding its three coordinates, with a fixed rule at ties. Then $|B_n+z_n|\le \sqrt{3}/2$. Put $\delta =\min (r/2,1/4)>0$. Conditional on the past, $B_{n+1}-B_n$ is a standard three-dimensional Gaussian vector. Its probability density function is bounded below on the open ball of radius $\delta$ centered at $-(B_n+z_n)$ byMultiplying by the volume of that open ball gives a constant $\epsilon >0$, independent of $n$ and the past, such thatThe probability of avoiding all lattice-centered radius-$r$ open balls at the first $N$ positive integer times is therefore at most $(1-\epsilon {)}^N$, by iterated conditional expectation. Eventually the path enters one such open ball almost surely. Since the initial point was outside that particular open ball, continuity forces an earlier crossing of its sphere. This proves Brownian hitting of lattice spheres:A uniform chance of hitting some member of the periodic family replaces recurrence to any one prescribed sphere; there is no contradiction with (b).

Work on a filtered probability space with the usual conditions for a filtration. A continuous martingale belongs to the class of L2-bounded continuous martingales whenThis means a uniform bound over the entire time interval, not merely being square-integrable at each individual time. The L2 martingale convergence theorem gives a terminal variable $M_{\infty }$ and almost sure convergence and convergence in the Lebesgue space $L^2$; moreover $M_t=\mathbb{E}[M_{\infty }\mid {\mathcal{F}}_t]$ by conditional expectation.

The predictable sigma-algebra $\mathcal{P}$ on $\Omega \times [0,\infty )$ is the smallest sigma-algebra making every left-continuous adapted process measurable. Equivalently, it is generated bywhere the sample coordinate comes first. A previsible process is precisely a process measurable for this predictable sigma-algebra. Values at time zero matter for its definition, although they make no contribution to integration against a continuous quadratic variation starting at zero.

A simple predictable process has the form $H_s={\sum }_{j=0}^{m-1}{h}_j{1}_{(t_j,t_{j+1}]}(s)$, where $0=t_0<\cdots <t_m<\infty$ are deterministic and $h_j$ is bounded and ${\mathcal{F}}_{t_j}$-measurable. A possible ${\mathcal{F}}_0$-measurable value at zero is irrelevant to this Itô integral. DefineEach coefficient is known before the corresponding increment. The conditional expectation of each subsequent increment is zero, so the result is a continuous martingale starting at zero. It is an L2-bounded continuous martingale because it is a finite sum of bounded coefficients times stopped increments of an L2-bounded continuous martingale. This definition is independent of the chosen subdivision: splitting an interval simply splits its increment into a telescoping sum. General admissible predictable processes are then integrated by completion using the Itô isometry; unbounded step coefficients are allowed when the weighted $L^2$ condition in the quadratic-variation measure in the next part holds.

Fix a L2-bounded continuous martingale $M$. Its quadratic-variation measure on the predictable sigma-algebra isIt is finite, since ${\nu }_M(\Omega \times [0,\infty ))=\mathbb{E}[M{]}_{\infty }=\mathbb{E}{M}_{\infty }^2-\mathbb{E}{M}_0^2$. The source Hilbert space is $L^2(M)=L^2(\Omega \times [0,\infty ),\mathcal{P},{\nu }_M)$: predictable processes with finite $\mathbb{E}\int H^{2}d[M]$, identified when equal ${\nu }_M$-almost everywhere. Its norm is $\Vert H{\Vert }_{L^2(M)}^2=\mathbb{E}\int H^{2}d[M]$.

The target ${\mathcal{M}}_0^2$ consists of L2-bounded continuous martingales starting at zero, identified up to indistinguishability of stochastic processes, with norm $\Vert K{\Vert }_{{\mathcal{M}}_0^2}=\Vert K_{\infty }{\Vert }_2$. It is a Hilbert space: terminal values belong to the closed linear subspace of $L^2({\mathcal{F}}_{\infty })$ whose conditional expectation given ${\mathcal{F}}_0$ is zero and whose associated martingales have continuous versions. Closure of that continuous-version subspace follows from the Doob L2 maximal inequality and an almost surely uniformly convergent subsequence.

The Itô isometry says that integration extends uniquely from simple predictable processes to a linear isometry $L^2(M)⟶{\mathcal{M}}_0^2$, withFor every finite $t$, the same identity holds with the integral restricted to $[0,t]$ and the left side $\mathbb{E}|(H\cdot M{)}_t{|}^2$. No claim of surjectivity onto all of ${\mathcal{M}}_0^2$ is needed: that would require an additional Martingale representation theorem.

Put $S=H\cdot M$ and $A_t={\int }_0^t{H}_s^{2}d[M{]}_s$. The candidate $A$ is continuous, adapted, nondecreasing, starts at zero and has integrable terminal value. We prove $S^2-A$ is a martingale, rather than assuming the desired quadratic variation of a stochastic integral.

The assumed boundedness of $M$ also makes $M^2-[M]$ a true martingale on every finite horizon. Indeed, apply its local martingale identity at a localizing sequence; bounded stopped squares and the Fatou lemma show that $[M{]}_t$ is integrable. The stopped expressions are then dominated by ${\sup }_s|M_s{|}^2+[M{]}_t$, so the dominated convergence theorem gives the unrestricted conditional identity on that horizon.

First let $H$ be a bounded simple predictable process. On each interval with coefficient $h$, an increment of $S$ is $h(M_t-M_a)$. The defining quadratic variation identity for $M$, together with the martingale property of $M$, makesa martingale for $t\ge a$. Expanding $S_t^2-S_a^2$ gives a term $2S_{a}h(M_t-M_a)$ and the square term above; their conditional expectations yield the desired identity on that interval. Integrability follows because the variables are square-integrable; concatenating the finitely many intervals gives $S^2-A$ a martingale.

For general bounded predictable $H$, choose bounded simple predictable processes $H^n\to H$ in $L^2(M)$, which is possible since they generate the predictable sigma-algebra. Set $S^n=H^n\cdot M$ and $A_t^n={\int }_0^t(H_s^n{)}^{2}d[M{]}_s$. The Itô isometry and Doob L2 maximal inequality imply $S_t^n\to S_t$ in $L^2$; the Cauchy-Schwarz inequality gives $(S_t^n{)}^2\to S_t^2$ in $L^1$ andPass the conditional expectation identity for $(S^n{)}^2-A^n$ to the $L^1$ limit. Thus $S^2-A$ is a martingale. Uniqueness in the defining quadratic variation property now gives

Before the Loewner swallowing time $T_z$, put $Z_t=g_t(z)-U_t=X_t+i{Y}_t$ and ${\theta }_t=\arg Z_t\in (0,\pi )$. The Itô formula applied to the holomorphic logarithm in the complex upper half-plane givesTaking imaginary parts yields the SLE angle processFor $\kappa =4$ the drift vanishes, sois a continuous local martingale. Since $0<{\theta }_t<\pi$, every localized stopped version is a bounded true martingale. After resolving the lifetime issue below, the dominated convergence theorem applied to conditional expectations removes localization and gives a true bounded martingale.

For completeness, there is no finite lifetime ambiguity for a fixed interior $z$ in this parameter range. The preceding Bessel process and Conformal Markov property of SLE argument gives a simple trace staying in the interior, so no point is swallowed in a disconnected pocket. If $T_z$ were finite, the trace would have to reach $z$, forcing the Loewner conformal radius to tend to zero by the Koebe quarter theorem. ButThe Dambis-Dubins-Schwarz theorem says that a bounded continuous local martingale cannot have infinite quadratic variation before a finite terminal time: that would require a Brownian motion to stay in a bounded interval for all clock times. Therefore ${{\rm Y}}_t$ cannot tend to zero at a finite $T_z$, a contradiction. Thus $T_z=\infty$ almost surely for each fixed $z$. This proves the SLE4 angle martingale assertion: the angle is a bounded continuous martingale. In particular $\mathbb{E}[{\theta }_t]=\arg z$. The upper-half-plane branch of the argument is used throughout.

For EM for zero-inflated negative binomial regression, at iteration $k$ apply Bayes theorem to each zero in the zero-inflated negative binomial model. The E step computes the conditional expectation of its latent indicator:Treating these values as fixed, the M step maximizes the expected complete-data log-likelihood. Its two parameter blocks separate:The second block is a weighted negative binomial regression with fixed size $r$. Writing ${\eta }_i=x_i^T\beta$ and ${\lambda }_i=e^{{\eta }_i}$, its parameter-dependent objective isIts score function and negative Hessian matrix areThe objective is concave, so a finite maximizer, when it exists, can be found by Newton method or iteratively reweighted least squares. The latter uses working response ${\eta }_i+(y_i-{\lambda }_i)/{\lambda }_i$ and weights $w_{i}r{\lambda }_i/(r+{\lambda }_i)$. This is not an unweighted regression on only the positive counts: zeros retain fractional weight in the count component.

Start with $0<\pi <1$ and finite coefficients, alternate the two steps, and monitor the observed log-likelihood. Exact maximization makes the expectation-maximization algorithm nondecreasing in that likelihood, although the mixture may have multiple stationary points. All-zero data can lead to boundary fits and poor identification; multiple initializations and explicit checks for boundary parameters are useful.

Let ${\mathcal{F}}_t$ contain observations through time $t$. In the Gaussian moving-average process of order one, the innovations inare independent of ${\mathcal{F}}_t$. Hence the conditional expectation is the constant mean. For known parameters, the forecast and its mean squared prediction error areThe mean prediction error itself is zero; the requested uncertainty is naturally interpreted as expected squared error. Plugging in the fit gives forecast $15.0133$, mean squared error $6.731002$, and root mean squared error about $2.5944$.

The usual naive time-series forecast repeats $Y_t$. Its unconditional expected squared error issince $\gamma (7)=0$. ThusConditional on ${\mathcal{F}}_t$, the naive error is $\gamma (0)+(Y_t-\mu {)}^2$, explaining directly why the conditional-mean forecast is better under squared-error loss. These are fitted-model calculations that ignore parameter-estimation uncertainty. Estimating the unknown mean from the past adds its estimation mean squared error to the seven-step forecast risk; the printed summary does not provide a complete assessment of that extra uncertainty. This is distinct from claiming a signed mean error as a measure of forecasting accuracy.

Use Bayes theorem with the individual survivor function. Writing $z=\theta t^2/2$, the frailty distribution among survivors has densityHenceThe conditional expectation starts at one and declines toward zero when $\theta >0$. Survival favors the less frail individuals, precisely explaining why the population hazard under exponential frailty equals $h_0(t)$ times a decreasing multiplier. The original PDF asks for conditioning on the event $T>t$; the TeX's misplaced inequality is a transcription error, not conditioning on an observed exact event time.

Put $L={\sup }_x\Vert \nabla f(x){\Vert }_{\infty }$, the uniform bound on the supremum norm of the gradient. The fundamental theorem of calculus on a line segment, together with the inner product inequality $|⟨a,b⟩|\le \Vert a{\Vert }_{\infty }\Vert b{\Vert }_1$, givesThus $f$ is Lipschitz continuous with respect to the L1 norm. The Gaussian random variables comprising $X$ and $Y$ have finite exponential moments of their absolute values. More explicitly, for $c\ge 0$, Hölder's inequality gives $\mathbb{E}{e}^{c\Vert X{\Vert }_1}\le {\prod }_{j=1}^n(\mathbb{E}{e}^{nc|X_j|}{)}^{1/n}<\infty$. Consequently $\mathbb{E}f(X)$ and every exponential expression below are finite, including when the covariance matrix is singular.

Use the convex function $\Psi (v)=e^{\lambda |v|}$. Since $Y$ is an independent copy of $X$, Jensen inequality for the conditional expectation givesFor $0\le \theta \le \pi /2$, define the Gaussian rotation of independent copiesWriting $\Sigma =\mathrm{Cov}(X)$, both rotated covariance matrices are $\Sigma$ and their cross-covariance is zero. The pair has a centered multivariate normal distribution, so its components are independent and $(U_{\theta },V_{\theta })$ has the same probability law as $(X,Y)$. This calculation uses no inverse of $\Sigma$ and therefore also proves the assertion for degenerate multivariate normal distributions.

The path $\theta \mapsto U_{\theta }$ runs from $Y$ to $X$. The chain rule and the fundamental theorem of calculus give the pathwise identityApply Jensen inequality to the uniform probability measure on $[0,\pi /2]$. Then use Tonelli theorem and the Gaussian rotation of independent copies to obtainThis proves the Gaussian rotation interpolation inequality in the required form:The stated almost-sure domination now gives an exponential moment of an absolute standard normal variable bound. If $a=\lambda \pi /2$, completing the square in the standard normal density yieldswhere $\Phi$ is the standard normal distribution function. Hence Markov inequality gives, for every $\lambda >0$,The exponent is minimized at $\lambda =4u/{\pi }^2>0$. Therefore the resulting Gaussian tail bound isNo independence of $Z$ from $X$ or $Y$ is needed: only the given almost-sure domination is used.

The bounded-time optional stopping theorem states that if $X$ is an integrable discrete-time martingale and $\tau$ is a stopping time bounded by a deterministic integer $N$, then $X_{\tau }$ is integrable and $\mathbb{E}{X}_{\tau }=\mathbb{E}{X}_0$. Indeed $|X_{\tau }|\le {\sum }_{j=0}^N|X_j|$ proves integrability, andThe event $\{\tau \ge j\}=\{\tau >j-1\}$ belongs to ${\mathcal{F}}_{j-1}$. Conditional expectation of each summand given ${\mathcal{F}}_{j-1}$ is zero, soMore generally, if $\sigma \le \tau \le N$ are stopping times, $\mathbb{E}[X_{\tau }\mid {\mathcal{F}}_{\sigma }]=X_{\sigma }$. To prove this, test against $A\in {\mathcal{F}}_{\sigma }$ and apply the same argument to $1_A{1}_{\{\sigma <j\le \tau \}}(X_j-X_{j-1})$; its coefficient is ${\mathcal{F}}_{j-1}$-measurable. Boundedness of the stopping times makes the sums finite, so no limiting optional-stopping hypothesis is being omitted.

Let ${\xi }_0^{n-1}={\bigvee }_{j=0}^{n-1}{T}^{-j}\xi$ denote the join of measurable partitions recording the first $n$ observations, and set $H_n=H_{\mu }({\xi }_0^{n-1})$, with $H_0=0$. The entropy of a countable measurable partition is $H_{\mu }(\xi )=-{\sum }_{A\in \xi }\mu (A)\log \mu (A)$, using natural logarithms and $0\log 0=0$. Null atoms can be discarded. Throughout, $\mu$ is a probability measure.

The stronger property needed for monotonicity of normalized block entropy is that its increments decrease. DefineSince conditioning reduces entropy, $c_{k+1}\le c_k$. The conditional entropy chain rule and measure preservation giveIt follows that $c_n\le H_n/n$, and consequentlyThis uses stationarity as well as the entropy chain rule; subadditivity alone would not establish monotonicity of every successive ratio.

The entropy rate of a measurable partition and the Kolmogorov-Sinai entropy are, respectively,One may equivalently take the supremum over countable finite-entropy measurable partitions.

The Shannon-McMillan-Breiman theorem states that for a countable measurable partition with $H_{\mu }(\xi )<\infty$, the normalized informationconverges almost everywhere and in $L^1$ to a $T$ invariant function $h_{\xi }(x)$ with integral $h_{\mu }(T,\xi )$. Here ${\xi }_0^{n-1}(x)$ is the atom containing $x$. If $T$ is an ergodic transformation, then $h_{\xi }(x)=h_{\mu }(T,\xi )$ almost everywhere. We prove the general form, including an explicit formula for its limit.

Let ${\mathcal{F}}_k$ be the sigma-algebra generated by ${\bigvee }_{j=1}^k{T}^{-j}\xi$, let ${\mathcal{F}}_0$ be trivial, and put ${\mathcal{F}}_{\infty }={\bigvee }_{j=1}^{\infty }{T}^{-j}\sigma (\xi )$. For every atom $A$, the probabilitiesform a bounded conditional-expectation martingale. The Martingale convergence theorem gives $p_k^A\to p_{\infty }^A={\mathbb{E}}_{\mu }[1_A\mid {\mathcal{F}}_{\infty }]$ almost everywhere and in $L^1$. Each of these probabilities is positive almost everywhere on $A$: for example, integrating $1_A$ over the ${\mathcal{F}}_{\infty }$ measurable set $\{p_{\infty }^A=0\}$ gives zero. Countability of $\xi$ lets us choose a common full-measure set for every atom.

The conditional information functionstherefore converge almost everywhere to $f_{\infty }=I_{\mu }(\xi \mid {\mathcal{F}}_{\infty })$. We need an integrable bound on this sequence; convergence of the probabilities alone would not supply one after taking logarithms. The allowed maximal inequality for conditional information functions gives, for $F={\sup }_{k\ge 0}{f}_k$,Combining this with the bound by $\mu (A)$ and the tail integral formula for moments yieldsThus $0\le f_{\infty }\le F$, and the dominated convergence theorem gives $f_k\to f_{\infty }$ in $L^1$. In particular,because $H_n/n$ is the average of the decreasing sequence $(c_k)$.

The information chain rule, applied from the final observation backwards, gives the exact identityIndeed, the symbol at time $j$ is conditioned on times $j+1,\dots ,n-1$; pulling that conditional probability back by $T^j$ gives $f_{n-1-j}\circ T^j$. Measure preservation is sufficient for this pullback identity.

We now prove the required triangular ergodic averaging lemma in this instance. Put $g_M={\sup }_{k\ge M}|f_k-f_{\infty }|$. Then $g_M\downarrow 0$ almost everywhere, $g_M\le 2F$, and $\Vert g_M{\Vert }_1\to 0$. Split the difference between the displayed triangular sum and ${\sum }_{j=0}^{n-1}{f}_{\infty }(T^{j}x)$. Terms whose index is at least $M$ are bounded by ${\sum }_{j=0}^{n-1}{g}_M(T^{j}x)$. The remaining terms are at mostAfter division by $n$, each of these finitely many end terms tends to zero almost everywhere by the linear growth bound for integrable observables proved in Question 1. Applying the Birkhoff ergodic theorem to $g_M$, for every fixed $M$, therefore givesThese conditional expectations decrease to zero almost everywhere: they decrease, and their integrals $\int g_{M}d\mu$ tend to zero. Letting $M\to \infty$, then applying the Birkhoff ergodic theorem to $f_{\infty }$, proves

For $L^1$ convergence, measure preservation gives the direct estimateCombine this with $L^1$ convergence of the ergodic averages of $f_{\infty }$. Finally, $\int h_{\xi }d\mu =\int f_{\infty }d\mu =h_{\mu }(T,\xi )$. In the ergodic case the invariant sigma-algebra is trivial, completing the familiar form of the theorem:

Write $\Delta X_{n+1}=X_{n+1}-X_n$ and $Y={\sum }_{n\ge 0}|\Delta X_{n+1}|1_{\{T>n\}}$. Since $T$ is a stopping time, $\{T>n\}\in {\mathcal{F}}_n$. The Tonelli theorem, conditional expectation, and (c)'s tail-sum formula for expectation yieldThus $Y$ is an integrable random variable. Since $X_0=0$, telescoping gives $|X_T|\le Y$ and $|X_n|1_{\{T>n\}}\le Y{1}_{\{T>n\}}$. The dominated convergence theorem gives $\mathbb{E}[|X_n|1_{\{T>n\}}]\to 0$, so (b) gives uniform integrability of the stopped martingale.

The optional stopping theorem at the bounded stopping time $n\wedge T$ gives $\mathbb{E}{X}_{n\wedge T}=\mathbb{E}{X}_0=0$. Passing to the limit using the convergence in L1 proved in (b),One can also pass directly by the dominated convergence theorem, since $|X_{n\wedge T}|\le Y$.

Let ${\mathcal{G}}_n=\sigma (S_n,S_{n+1},\dots )$. Since $X_{n+j}=S_{n+j}-S_{n+j-1}$,Permuting $X_1,\dots ,X_n$ preserves their joint probability distribution and leaves every generator of ${\mathcal{G}}_n$ unchanged. Therefore, for each bounded ${\mathcal{G}}_n$-measurable $H$,This is the symmetry of exchangeable random variables. By the defining identity of conditional expectation, all $\mathbb{E}[X_j\mid {\mathcal{G}}_n]$ are equal almost surely. Summing these conditional expectations and using that $S_n$ is ${\mathcal{G}}_n$-measurable givesHence the conditional expectation of a summand given future partial sums isThe integrability of each $X_j$ justifies every conditional expectation above.