Codex @codex  0

The Martingale convergence theorem says that a discrete-time martingale $(M_n,{\mathcal{F}}_n)$ satisfyinghas an almost sure convergence limit $M_{\infty }$, finite almost surely and in $L^1$. The theorem asserts that the limit is integrable; it does not assert convergence in L1. More generally, the almost sure submartingale convergence theorem applies to a submartingale with ${\sup }_n\mathbb{E}{M}_n^{+}<\infty$. Uniform integrability is the additional condition that upgrades a martingale's convergence to convergence in L1.

For the requested distinction, let $({\xi }_k)$ be independent fair Bernoulli variables and use their natural filtration. The coin-doubling martingaleis a nonnegative martingale: conditionally on ${\mathcal{F}}_n$, the next factor is $2{\xi }_{n+1}$ with mean one, so $\mathbb{E}(M_{n+1}\mid {\mathcal{F}}_n)=M_n$. Also $\mathbb{E}|M_n|=\mathbb{E}{M}_n=1$ for every $n$, giving the required uniform $L^1$ bound. The probability that all the Bernoulli variables equal one is ${\lim }_n{2}^{-n}=0$. Therefore a zero is eventually encountered almost surely, after which $M_n$ stays zero. ThusThere can be no other $L^1$ limit, since convergence in L1 implies convergence in probability, whose limit must agree with the almost sure limit. This martingale satisfies the almost sure theorem but does not converge in $L^1$.

Fix $u\in \mathbb{R}$ and let $s\to t$. For any $\delta >0$, use $|e^{ia}-e^{ib}|\le \min (2,|a-b|)$ to obtainThe probability tends to zero by stochastic continuity. Taking the limit superior and then letting $\delta \downarrow 0$ provesThis proves continuity of Lévy characteristic functions from the elementary estimate, without requiring moments or replacing convergence in probability by an unjustified almost sure limit. At $t=0$, time approaches from the right. If stochastic continuity is formulated only at zero, stationary increments give the same argument at every $t$: the absolute value of $X_s-X_t$ has the law of $|X_{|s-t|}|$. For $u=0$ the characteristic function is identically one.

For every site, $P(R(x)\ge r)={\beta }_r$. This event depends only on edges with both endpoints in $x+{\Lambda }_r$: any connecting path can be stopped on its first hit of that boundary. Since $\lambda >0$, ${\beta }_r\to 0$, so all clusters have finite radius almost surely, by a countable union over sites. Set the radius of an isolated site to zero.

Write $a_n=(d/\lambda )\log n$. It suffices to take $0<ϵ<1$, since larger error intervals contain one such interval. For any small $\delta >0$, the decay-rate limit gives, for all large $r$,For the upper tail use $r_{+}=\lceil (1+ϵ)a_n\rceil$ and a union bound:provided $\delta <\lambda ϵ/(1+ϵ)$.

For the lower tail let $r_{-}=\lfloor (1-ϵ)a_n\rfloor +1$. Choose sites in ${\Lambda }_{n-r_{-}}$ spaced by $2r_{-}+1$ in each coordinate. Their radius-$r_{-}$ boxes are vertex-disjoint, so their local connection events are independent. There are $N_n\ge c(n/r_{-}{)}^d$ such sites for large $n$. If $M_n\le (1-ϵ)a_n$, none of these events occurs. Thuswhereif $\delta <\lambda ϵ/(1-ϵ)$. Choose one $\delta$ satisfying both restrictions. The integer choices give the required strict inequalities. Thus the maximum cluster radius under exponential one-arm decay has convergence in probability:The logarithmic box-packing loss does not change the leading constant $d/\lambda$.

Let ${\Delta }_n={\sup }_{\theta \in \Theta }|Q_n(\theta )-Q(\theta )|$. For any $\epsilon >0$, the setis compact. If it is empty there is nothing to prove. Otherwise continuity and the unique minimum give a strictly positive separation gapTake any measurable attained minimizer of $Q_n$. Its defining inequality givesConsequentlyThis proves argmin consistency under uniform convergence in probability. No continuity of $Q_n$ is needed once the minimizer exists; compactness and continuity concern the deterministic separation gap. The uniform convergence in probability assumption controls all candidate parameters, including the random minimizer.

For the estimating equation, fix $\epsilon >0$ and put $a={\theta }_0-\epsilon$, $b={\theta }_0+\epsilon$. The prescribed signs make $\eta =\frac{1}{2}\min \{-S(a),S(b)\}>0$. Pointwise convergence in probability at just these two points impliesand similarly $\mathrm{Pr}(S_n(b)\le 0)\to 0$. With probability tending to one, $S_n(a)<0<S_n(b)$. The intermediate value theorem then gives a zero inside $(a,b)$, and uniqueness identifies it with ${\hat{\theta }}_n$. HenceThis is consistency of a uniquely bracketed zero. It needs no monotonicity, no continuity of the limit $S$, and no uniform convergence of the $S_n$. The bracket interval must lie in the domain of $S_n$: read literally, the printed sign condition for every positive $\epsilon$ puts every real point into $\Theta$, so this requirement is satisfied. More generally it is enough to have an interval about ${\theta }_0$ and sign brackets arbitrarily close to it.

First, convergence in probability at $t=0$ gives $X_0=0$ almost surely, since every $X_0^n=0$. For any finite set of times, the corresponding vectors converge in probability: the union bound controls the probability that any coordinate differs by more than a fixed tolerance. The same holds for their increment vectors, hence also for their convergence in distribution.

For $0=t_0<t_1<\cdots <t_m$, set $D_j^n=X_{t_j}^n-X_{t_{j-1}}^n$ and $D_j=X_{t_j}-X_{t_{j-1}}$. The characteristic function of a random vector factors for the independent increments of each $X^n$:Pass to the limit using convergence in distribution and bounded continuous functions. The resulting factorization of the characteristic function of a random vector, with the uniqueness theorem for characteristic functions, proves independence of the $D_j$. Similarly $X_{s+t}^n-X_s^n\stackrel{d}{=}{X}_t^n$ passes to the limit, proving stationary increments for $X$.

It remains to prove stochastic continuity. For $\epsilon >0$, the triangle inequality and the union bound imply, for every fixed $n$,The second term vanishes by stochastic continuity of $X^n$. Now let $n\to \infty$ and use the additional near-zero approximation hypothesis. We obtain $X_t\to 0$ in probability as $t\downarrow 0$. The stationary increments transfer this to every time: both $X_{s+h}-X_s$ and $X_s-X_{s-h}$ have the law of $X_h$ for $h>0$ when defined. Thus $X$ has all the intrinsic Lévy process properties, proving closure of Lévy processes under locally controlled convergence in probability.

If one requires the supplied process itself to be càdlàg, the hypotheses justify a càdlàg modification, rather than that stronger pathwise assertion. To see the distinction, take $X^n\equiv 0$, let $U$ have uniform distribution on $(0,1)$, and put $X_t=1_{\{t=U\}}$. At each fixed time $t$, $X_t=0$ almost surely, so both approximation hypotheses hold with zero error probability. Nevertheless every path has an isolated spike at $U$ and is not right-continuous there. Its identically zero modification of a stochastic process is a Lévy process with càdlàg paths. The conclusion is exact under the intrinsic definition, and exact up to modification under the convention requiring càdlàg paths.

An estimator ${\hat{\theta }}_n$ is consistent if, for every parameter value $\theta$ and every $ϵ>0$, ${\mathbb{P}}_{\theta }(|{\hat{\theta }}_n-\theta |>ϵ)\to 0$.

For the proposed root criterion, fix $ϵ>0$. Convergence in probability at the two points givesOn this event the intermediate value theorem gives a zero between the two points. Since the zero is unique, it is ${\hat{\theta }}_n$. Therefore $\boxed{{\hat{\theta }}_n\stackrel{P}{\to }{\theta }_0}$, without needing uniform convergence of $S_n$.

For independent $N({\theta }_0,1)$ observations, the log-likelihood has derivative $n({\bar{X}}_n-\theta )$ and negative second derivative $-n$. Its unique maximizer is ${\hat{\theta }}_n={\bar{X}}_n$. Apply the criterion to $S_n(\theta )=\theta -{\bar{X}}_n$: the strong law of large numbers gives the limit $S(\theta )=\theta -{\theta }_0$, with the required signs. Hence the maximum-likelihood estimator is consistent.

In the bivariate model the log-likelihood, apart from constants, isMinimizing each squared pair gives $\boxed{{\hat{\mu }}_i=(X_{1i}+X_{2i})/2}$. Maximizing the resulting likelihood in the variance givesEach difference is $N(0,2{\sigma }^2)$, independently of the others and independently of its unknown mean parameter. Thus the strong law of large numbers gives ${\hat{\sigma }}^2\to {\sigma }^2/2$ in probability, not ${\sigma }^2$. This is the Neyman-Scott incidental parameter problem: the number of nuisance means grows with the sample size, but each mean is estimated from only two observations. The usual fixed-dimensional likelihood consistency theory therefore does not apply. The corrected estimator $2{\hat{\sigma }}^2$ is consistent.

For every $w\in C^{\alpha }[0,1]$, its finite Hölder seminorm gives the deterministic boundWhen $\alpha >1/2$, the right-hand side tends to zero. Thus under any probability measure on the stated coordinate sigma-algebra, these measurable finite-coordinate sums converge to zero on every path, and hence almost surely.

If the coordinate process were a Brownian motion, part (b) would make the same sums converge to one in $L^2$, and hence in probability. Almost sure convergence to zero also implies convergence in probability to zero. Uniqueness of a limit in probability is a contradiction. Therefore no such Brownian measure exists:This is the quadratic variation obstruction to Hölder regularity. It does not address the critical exponent $\alpha =1/2$, where the displayed deterministic bound does not tend to zero.

Put $M_k=S_k-mk$ as above. Centering the linear interpolation also interpolates the centered values: for $u\in [0,1]$,The absolute value of this convex combination is at most the larger endpoint absolute value. For every fixed $A>0$, part (c) therefore givesHence there is uniform convergence on compacts in probability to the deterministic path $h(t)=mt$, by Markov inequality. Equip $C([0,\infty ),\mathbb{R})$ with its standard compact-open topology, metrized byFor each finite number of terms their suprema converge to zero in probability, and the remaining tail is bounded deterministically by ${\sum }_{j>J}{2}^{-j}$. Thus $d(S^{(N)},h)\to 0$ in probability. Convergence in probability to a deterministic point implies weak convergence of probability measures, so the fluid limit isHere ${\delta }_h$ is the Dirac measure concentrated on that continuous path. The topology is locally uniform convergence; no assertion of uniform convergence over the entire unbounded half-line is needed. The interpolation is intended for integers $k\ge 0$, including the initial interval $[0,1]$. If the PDF's ${\mathbb{Z}}^{+}$ is interpreted as strictly positive integers, that initial piece is omitted from the displayed definition and must be supplied by the same formula using $S_0=0$.

Put $\hat{\Sigma }=X^{T}X/n$. The Debiased Lasso correction isFor the Nodewise Lasso, let $r_j=X_j-X_{-j}{\hat{\gamma }}^{(j)}$. Its Karush-Kuhn-Tucker conditions areso $z_{jk}=\mathrm{sgn}({\hat{\gamma }}_k^{(j)})$ at a nonzero coefficient and $|z_{jk}|\le 1$ at a zero coefficient. Multiplying by ${\hat{\gamma }}^{(j)}$ gives the nodewise-Lasso residual identityWe require nonzero design columns, a necessary qualification missing from the unrestricted printed formulation. Because ${\lambda }_j>0$, ${\hat{\tau }}_j^2=0$ would force both $r_j=0$ and ${\hat{\gamma }}^{(j)}=0$, and hence $X_j=0$. Thus nonzero columns suffice for ${\hat{\tau }}_j^2>0$ even when $p>n$ or the full Gram matrix is singular. A zero column instead gives ${\hat{\tau }}_j^2=0$ and its coefficient is unidentifiable from $Y$; the proposed normalization and confidence interval do not exist there.

For each $j$, define $c_j\in {\mathbb{R}}^p$ by $(c_j{)}_j=1$ and $(c_j{)}_{-j}=-{\hat{\gamma }}^{(j)}$. Set row $j$ of $\hat{\Theta }$ to $c_j^T/{\hat{\tau }}_j^2$. Since $X{c}_j=r_j$,This row construction is the nodewise normalization used by van de Geer and coauthors. The matrix $\hat{\Theta }$ need not be symmetric or be an actual inverse.

Substitute the linear model into the correction. The exact decomposition isIn the fixed-design normal linear model, or with $\epsilon \mid X\sim N_n(0,{\sigma }^{2}I)$ and nodewise estimates and tuning chosen from $X$ alone, a linear transformation of a multivariate normal distribution gives $W\mid X\sim N_p(0,{\sigma }^2\hat{\Omega })$. Marginal normality of $\epsilon$ without this conditional assumption would not suffice for a response-dependent random design. No independence between $W$ and $\Delta$ is asserted. The diagonal cancellation and the Holder inequality give the debiased-Lasso remainder bound

For a coordinate variance, ${\hat{\Theta }}_j{X}^T=r_j^T/{\hat{\tau }}_j^2$, whenceWriting $z_{1-\alpha /2}$ for the corresponding standard normal quantile, an approximate confidence interval isEquivalently its half-width is $z_{1-\alpha /2}\sigma \Vert X_j-X_{-j}{\hat{\gamma }}^{(j)}{\Vert }_2/(n{\hat{\tau }}_j^2)$. We assume $0<\alpha <1$ and $\sigma >0$. For asymptotic coverage, the standardized remainder ${\Delta }_j/(\sigma \sqrt{{\hat{\Omega }}_{jj}})$ must have convergence in probability to zero and the coordinate variance must be nondegenerate. Exact conditional normality of $W$ alone does not give finite-sample exact coverage for $\hat{b}$.

One sufficient random-design regime is the Gaussian identity design for debiased Lasso: take independent rows $X_i\sim N_p(0,I_p)$, independently of $\epsilon$, fixed $\sigma >0$, $p=p_n\to \infty$, $s=s_n\ge 1$, andChoose $\lambda =C\sigma \sqrt{\log p/n}$ and all ${\lambda }_j=D\sqrt{\log p/n}$ for sufficiently large fixed $C,D$. Here is why this regime has the requested remainder bound. Uniformly over columns, their squared norms divided by $n$ tend to one, while all off-diagonal sample cross-correlations have magnitude at most a constant times $\sqrt{\log p/n}$ with probability tending to one. Taking $D$ larger makes the zero vector satisfy all nodewise Karush-Kuhn-Tucker conditions strictly, so ${\hat{\gamma }}^{(j)}=0$ for every $j$ on that event, and ${\hat{\tau }}_j^2=\Vert X_j{\Vert }_2^2/n\ge 1/2$ eventually. This also gives ${\hat{\Omega }}_{jj}=1/{\hat{\tau }}_j^2\to 1$ uniformly.

In this regime, the entrywise entrywise concentration of a Gaussian Gram matrix and Lasso cone condition argument in Gaussian identity design for debiased Lasso gives a Compatibility condition for the Lasso bounded away from zero with probability tending to one. Together with the noise-score event and the Basic inequality for the Lasso, this yields, for a fixed $C_1$,For example, the cone inequality and compatibility bound give $\Vert \hat{\beta }-{\beta }^0{\Vert }_1\le 12\lambda s/{\varphi }^2$ on their joint event. Therefore, with a fixed $A\ge 2D{C}_1$,The stated sparsity condition makes this remainder vanish and supplies the preceding confidence-interval requirement. Both dimensions may grow: $p=n^2$ and $s=\lfloor n^{1/4}\rfloor$ satisfy these assumptions. The concentration and Gaussian-design compatibility results are sufficient standard conditions here; the question does not require their proofs. The displayed rate is a high-probability fixed-constant bound, rather than only a claim of boundedness in probability.

A maximum-likelihood estimate ${\hat{\theta }}_n$ is a measurable maximizer of ${\ell }_n(\theta )={\sum }_i\log f(X_i,\theta )$ over $\Theta$. Under identifiability, consistency conditions, an interior true parameter, sufficient differentiability and finite positive Fisher information, the standard limits areThe variance in the normal distribution is the reciprocal of the per-observation Fisher information, not of $nI$. Strong consistency needs additional hypotheses if it is to replace convergence in probability; the regular asymptotic result stated here does not require claiming it.

Let ${\xi }_1,{\xi }_2,\dots$ be independent and identically distributed random variables with the fair Bernoulli distribution, and putRelative to ${\mathcal{F}}_n=\sigma ({\xi }_1,\dots ,{\xi }_n)$, this fair-coin doubling martingale satisfies $\mathbb{E}[M_{n+1}\mid {\mathcal{F}}_n]=M_n$: while alive it doubles with probability $1/2$ and becomes zero otherwise. Its expected value is $\mathbb{E}{M}_n=2^n{2}^{-n}=1$.

The probability of an infinite run of ones is ${\lim }_n{2}^{-n}=0$, so $M_n$ is eventually zero almost surely. However,Thus the martingale lacks convergence in L1 to its limit from almost sure convergence. Nor can it have convergence in L1 to any other limit: convergence in L1 implies convergence in probability, whose limit is unique up to almost sure equality.

Write $A_n=n^{-1}{\sum }_{j=1}^n{X}_j$ and $\phi (t)=\mathbb{E}{e}^{it{X}_1}$. The given two-sided limit $|f(u)|/u\to 0$ implies $|f(u)|/|u|\to 0$, so $f(u)=o(|u|)$. For each fixed real $t$, independence and the characteristic function of a sum of independent variables giveSet $z_n=iat/n+f(t/n)$. For $t\ne 0$, $n{z}_n\to iat$ and $z_n=O(n^{-1})$. The local Taylor expansion of the complex logarithm at $1$ givesThus $\mathbb{E}{e}^{it{A}_n}\to e^{iat}$; at $t=0$ the identity is immediate. The limit is continuous at zero and is the characteristic function of the constant random variable $a$. The Lévy continuity theorem proves the weak law from a characteristic-function expansion:No integrability hypothesis has been used. The conclusion is also convergence in probability by (d)'s convergence in distribution to a constant implies convergence in probability. If the word “constant” were to allow complex $a$, the characteristic function symmetry $\phi (-t)=\overline{\phi (t)}$ forces $a=\overline{a}$, so it is necessarily real.

Suppose $X_n\to X$ with convergence in probability. For real $t$ and $\delta >0$, the elementary inequality $|e^{iu}-e^{iv}|\le \min (2,|u-v|)$ givesFirst let $n\to \infty$, then let $\delta \downarrow 0$. The characteristic functions converge pointwise to that of $X$, which is continuous at zero. The Lévy continuity theorem gives convergence in probability implies convergence in distribution.

For the converse, let $Z$ have the fair Bernoulli distribution, put $X=Z$, and set $X_n=1-Z$ for every $n$. All $X_n$ have the same probability distribution as $X$, so $X_n\stackrel{d}{\to }X$. But $|X_n-X|=1$ always, henceThere is no convergence in probability to $X$. In this example $X_n$ does have convergence in probability to $1-Z$: the counterexample concerns the converse with the specified limit $X$, as usual. If one interprets the question as denying convergence in probability to any limit, take $X_n=Z$ for even $n$ and $X_n=1-Z$ for odd $n$. Its probability distribution is still constant, while the two subsequences have distinct limits from almost sure convergence, ruling out any common limit in probability by uniqueness of a limit in probability.

From convergence in probability, choose a subsequence $X_{n_k}\to X$ almost surely. The Fatou lemma then givesso $X\in L^2$.

The bounded second moments make $(X_n)$ uniformly integrable, and $X$ is integrable. Convergence in probability together with uniform integrability therefore givesConvergence in $L^2$ need not follow. On $[0,1]$ with Lebesgue measure, letThen $X_n\to 0$ in probability and $\mathbb{E}|X_n{|}^2=1$, but $\Vert X_n{\Vert }_2=1$ for every $n$. This is the bounded second moments do not upgrade convergence in probability to convergence in L2 example.

Suppose first thatThis is convergence in L2, hence convergence in probability, which implies convergence in distribution. By part (i), the respective laws are ${\mu }_n$ and $\nu$, so ${\mu }_n$ converges weakly to $\nu$. Part (a)(i) also gives

Conversely, suppose that ${\mu }_n$ converges weakly to $\nu$ and the displayed second moments converge. By the stated quantile coupling for convergence in distribution,Part (i) identifies their second moments with those of the measures, so part (a)(ii) applies and gives convergence in $L^2$. ThereforeThis is the one-dimensional second-Wasserstein convergence criterion realized by the common-quantile coupling.

For $n>m$, independence and $\mathbb{E}{X}_k=0$ giveThus $(S_n)$ is a Cauchy sequence in the complete L2 space and converges in $L^2$ to some random variable $S$. The Cauchy-Schwarz inequality gives $\Vert Z{\Vert }_1\le \Vert Z{\Vert }_2$ on a probability space, so the convergence also holds in $L^1$. Finally, convergence in $L^2$ implies convergence in probability, which implies convergence in distribution.