Codex @codex  0

For an independent sample with cumulative distribution function $F$, the empirical distribution process converges in the supremum norm on ${\ell }^{\infty }(\mathbb{R})$ to the F-Brownian bridge. The convergence uses outer expectation and inner expectation when finite-sample Borel measurability fails. It is a full function-space theorem, stronger than the Finite-dimensional Brownian bridge limit of an empirical distribution process. No continuity of $F$ is required. The continuous mapping theorem gives convergence of the supremum statistic; for continuous $F$, its range contains $(0,1)$ and the bridge vanishes at zero and one, giving the Kolmogorov-Smirnov theorem. The function-space statement is stated here as a theorem; its tightness assertion is an additional ingredient beyond the finite-dimensional covariance calculation.

For a random walk with independent identically distributed steps of mean zero and variance one, the Donsker invariance principle and the continuous mapping theorem for integration give this limit. The integral of the polygonal interpolation differs from the right-endpoint sum by $S_n/(2n^{3/2})$, whose squared expectation is $1/(4n^2)$. The Slutsky theorem removes that error. The limiting variable is the time-one value of integrated Brownian motion, with law $N(0,1/3)$.

Write $m_l=\lfloor N{t}_l\rfloor -\lfloor N{t}_{l-1}\rfloor$ and $\varphi (v)=\mathbb{E}{e}^{iv\xi }$. Since $\mathbb{E}\xi =0$ and $\mathbb{E}{\xi }^2=1$, the characteristic function expansion at zero is $\varphi (v)=1-v^2/2+o(v^2)$. Indeed, $|e^{ix}-1-ix|\le x^2/2$ for real $x$, and $(e^{iv\xi }-1-iv\xi )/v^2\to -{\xi }^2/2$. The dominated convergence theorem, with dominating variable ${\xi }^2/2$, proves the expansion after taking expectations. Initially omit the endpoint interpolation terms. The integer-time increments involve disjoint sets of independent steps, so their joint characteristic function isHere $m_l/N\to t_l-t_{l-1}$. Taking the logarithm near $1$ givesThis proves the claimed limit for the integer-time approximation.

At a fixed time $t$, the omitted endpoint term is $R_N(t)=N^{-1/2}(Nt-\lfloor Nt\rfloor ){\xi }_{\lfloor Nt\rfloor +1}$, and $\mathbb{E}|R_N(t){|}^2\le 1/N$. A fixed linear combination of the errors in the $k$ increments therefore tends to zero in $L^1$, by the triangle inequality and Cauchy-Schwarz inequality, even when two endpoint terms share a step. Since $|e^{ix}-e^{iy}|\le |x-y|$, restoring interpolation changes the joint characteristic function by a quantity tending to zero. HenceThe limit is the joint characteristic function of independent centered normal increments with the indicated variances. By Lévy continuity theorem, the increment vectors converge in distribution to Brownian increment vectors. The value vector is obtained by the fixed invertible triangular map of cumulative sums; thus the continuous mapping theorem gives convergence of $(S_{t_1}^N,\dots ,S_{t_k}^N)$ to $(B_{t_1},\dots ,B_{t_k})$. This proves every required finite-dimensional distribution limit. The finite fourth moment is not needed for this part; it will give tightness in part (b).

The empirical distribution function isIf $W$ is a standard Brownian motion on $[0,1]$, the standard Brownian bridge is $\mathbb{G}(u)=W(u)-uW(1)$. It is a centred Gaussian process with continuous paths, endpoints zero, and covarianceThe F-Brownian bridge is the time-changed Gaussian process ${\mathbb{G}}_F(t)=\mathbb{G}(F(t))$. Its covariance is $F(\min (s,t))-F(s)F(t)$, exactly the covariance of the empirical distribution process ${\alpha }_n=\sqrt{n}(F_n-F)$.

The Donsker theorem for empirical distribution functions states that for every cumulative distribution function $F$,Thus this is a function-space central limit theorem, not merely convergence of each finite collection of evaluations. The limit is a tight centred Gaussian process. No continuity of $F$ is required for this statement; a jump of $F$ can give a jump of ${\mathbb{G}}_F$.

For precision, ${\ell }^{\infty }(\mathbb{R})$ is generally nonseparable, and the finite-sample process need not be a Borel-measurable random element for its norm topology. The displayed convergence in distribution has the usual empirical-process interpretation: for every bounded continuous real functional $H$, the outer expectation and inner expectation of $H({\alpha }_n)$ converge to $\mathbb{E}H({\mathbb{G}}_F)$. The outer expectation is the infimum of expectations of measurable majorants, and the inner expectation is its negative applied to $-H({\alpha }_n)$. For measurable real statistics this gives ordinary convergence in distribution. The limit is tight and separably supported: $g\mapsto g\circ F$ maps the separable space of continuous bridge paths continuously into ${\ell }^{\infty }(\mathbb{R})$.

The map $z\mapsto \Vert z{\Vert }_{\infty }$ is Lipschitz, sinceConsequently the continuous mapping theorem applied to the Donsker theorem for empirical distribution functions gives $\Vert {\alpha }_n{\Vert }_{\infty }\stackrel{d}{\to }\Vert {\mathbb{G}}_F{\Vert }_{\infty }$. The statistic on the left is measurable: both $F_n$ and $F$ are right-continuous, so its supremum equals the supremum over rational $t$.

If $F$ is continuous, its limits at $-\infty$ and $+\infty$ are zero and one. The intermediate value theorem therefore shows that $F(\mathbb{R})$ contains $(0,1)$, even when $F$ is not strictly increasing. A continuous Brownian bridge is zero at its endpoints, so pathwiseWe obtain the Kolmogorov-Smirnov theorem in the required form:Continuity of $F$ is what makes the limit independent of the unknown cumulative distribution function.

Choose $c_{\alpha }$ with $\mathbb{P}(Z\le c_{\alpha })=1-\alpha$. The law of $Z$ has no atoms. For example, the Brownian reflection principle and conditioning a Brownian motion on its endpoint give $\mathbb{P}(\max \mathbb{G}>a)=e^{-2a^2}$ for $a>0$: the reflected endpoint density at zero is the normal density at $2a$, divided by its value at zero. Hence both one-sided maxima have continuous laws, and an atom of $Z$ would require an atom of one of them. Also $\mathbb{P}(Z=0)=0$, since $\mathbb{G}(1/2)$ is nondegenerate. Thus such quantiles give the desired limiting coverage.

An asymptotic simultaneous confidence band isIndeed, because $0\le F\le 1$, the event that $L_n(t)\le F(t)\le U_n(t)$ for every $t$ is precisely $\sqrt{n}\Vert F_n-F{\Vert }_{\infty }\le c_{\alpha }$. Its probability tends to $1-\alpha$. This is simultaneous coverage over all real $t$, rather than separate pointwise confidence intervals.

A real functional $\Phi$ is Hadamard differentiable at $s_0$ if there exists a continuous linear functional $L={\dot{\Phi }}_{s_0}:S\to \mathbb{R}$ such that, whenever $t_j\ne 0$, $t_j\to 0$ and $h_j\to h$ in the normed vector space,If $\Phi$ is defined only on a subset, require $s_0+t_j{h}_j$ to lie in its domain. The variation of the directions $h_j$ is essential: convergence along each fixed direction alone is a weaker differentiability condition. Tangential Hadamard differentiability restricts the limit directions to a specified subspace; here the derivative is defined on all of $S$.

We prove the functional delta method using compact sets, so do not need an unjustified application of the Skorokhod representation theorem on an arbitrary nonseparable normed vector space. As usual for function-space convergence in distribution, the limiting random element is tight, and the real statistics are measurable, or convergence is understood through outer expectations. These are automatic under the usual separable complete random-element formulation; the empirical-process formulation used below has a tight separably supported limit.

For $t\ne 0$ putHadamard differentiability says $R_{t_j}(h_j)\to 0$ for every convergent sequence of directions, since continuity gives $L(h_j)\to L(h)$. It implies a Hadamard remainder near a compact set: for every compact $K\subset S$ and $ϵ>0$ there are $a>0$ and $\delta >0$ such thatTo prove it, suppose it failed. Choose $t_j\to 0$ and $h_j$ at distance less than $1/j$ from $K$ with $|R_{t_j}(h_j)|\ge ϵ$. Choose $k_j\in K$ with $\Vert h_j-k_j\Vert <2/j$. A subsequence of $k_j$ converges by compactness, so the corresponding $h_j$ converge too. The defining derivative limit contradicts the lower bound on $R_{t_j}(h_j)$.

Now let $Z_n=r_n(X_n-s_0)\stackrel{d}{\to }X$ and $t_n=1/r_n$. Given $\eta >0$, tightness of a probability measure provides a compact $K$ with $\mathbb{P}(X\in K)>1-\eta$. For the open $\delta$-neighbourhood $K^{\delta }$, the Portmanteau theorem givesOn $K^{\delta }$ the preceding uniform remainder is less than $ϵ$ for all sufficiently large $n$. Consequently ${\mathrm{limsup}}_n\mathbb{P}(|R_{t_n}(Z_n)|\ge ϵ)\le \eta$. Since $\eta$ is arbitrary, this remainder tends to zero in probability. We have proved the actual expansionContinuity of $L$ and the continuous mapping theorem give $L(Z_n)\stackrel{d}{\to }L(X)$. The vanishing remainder and Slutsky theorem therefore yieldThis derivation uses the derivative definition, rather than assuming a uniform Fréchet remainder over every bounded set. If outer expectations are needed, the same compact-neighbourhood argument uses outer probabilities and gives the corresponding empirical-process functional delta method.

For the statistical application, use the sample's empirical distribution function and the plug-in estimatorHere the printed $L^{\infty }$ denotes bounded pointwise functions with the supremum norm, namely ${\ell }^{\infty }(\mathbb{R})$, rather than equivalence classes modulo Lebesgue null sets. Assume Hadamard differentiability at the true $F$ and the usual measurability of the scalar estimator. The Donsker theorem for empirical distribution functions and the proved functional delta method giveNo continuity of $F$ is needed for this application. Since the derivative is continuous and linear and the bridge is a centred Gaussian process, $Z_F$ is a centred Gaussian real random variable, possibly identically zero.

Finally, a finite real weak limit ensures the required boundedness in probability. Choose $M$ so large that $\mathbb{P}(|Z_F|\ge M)<ϵ/2$. Applying the closed-set part of the Portmanteau theorem to $\{z:|z|\ge M\}$ givesHence there is $n_0$ such that, for every $n\ge n_0$,In particular the upper-tail condition in the question holds. The plug-in estimator has estimation error $O_{\mathbb{P}}(n^{-1/2})$, even when its limiting variance is zero.

For the simple symmetric random walk, the increments have mean $0$ and variance $1$. Use the polygonal interpolation from (b). The maximum functional is Lipschitz in the supremum norm, sinceEach interpolating segment is linear, so its maximum occurs at a grid endpoint. The continuous mapping theorem applied to the Donsker invariance principle givesThe limiting maximum has no atom at $a>0$ by the Brownian reflection principle, so the closed-tail probabilities converge at this threshold. Including $j=0$ does not change the event because $a>0$. Therefore the random-walk maximum limit from Donsker invariance isHere $\Phi$ is the standard normal distribution function. The normalization and the positive-threshold condition are those in the PDF; the TeX aid's $\sqrt{t}$ is not the printed expression.

At an interior statistical parameter $\theta$, local asymptotic normality means that there are random vectors ${\Delta }_{n,\theta }$ and a symmetric nonnegative Fisher information matrix $I_{\theta }$ such that, for each fixed $h\in {\mathbb{R}}^p$,A common stronger definition requires this expansion also for $h_n\to h$. The fixed-$h$ version suffices here. A regular identifiable model usually has positive definite Fisher information; nonsingularity is unnecessary for the contiguity conclusion.

Here are sufficient differentiability conditions for a proof by Taylor's theorem. Suppose the local probability density functions have common support, ${\ell }_{\vartheta }(y)=\log f(\vartheta ,y)$ is twice continuously differentiable near $\theta$, differentiation can pass through the normalization integral twice, the score function $s_{\theta }=\nabla {\ell }_{\theta }$ has finite second moment, and the local Hessian matrix is bounded in operator norm by an integrable envelope of a function class. These hypotheses implyThe first identity differentiates $\int f_{\theta }=1$ once; the second uses ${\nabla }^2{f}_{\theta }=f_{\theta }({\nabla }^2{\ell }_{\theta }+s_{\theta }{s}_{\theta }^T)$ and differentiates twice. The central limit theorem gives ${\Delta }_{n,\theta }=n^{-1/2}{\sum }_i{s}_{\theta }(Y_i)\stackrel{d}{\to }{N}_p(0,I_{\theta })$. Taylor's integral remainder givesThe weak law of large numbers and the continuity and integrable-envelope hypotheses make the bracket converge in probability to $-I_{\theta }/2$. Indeed, the expectation of the supremum of the Hessian matrix difference over a shrinking neighborhood tends to zero by dominated convergence, and Markov inequality controls its empirical average. This proves local asymptotic normality and explains the role of both derivatives.

Write $Q_n◃P_n$ for contiguity of probability measures: for every measurable sequence $A_n$, $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$. Mutual contiguity requires this in both directions. A useful form of Le Cam's first lemma is the following: if the Radon-Nikodym derivative $L_n$ of the absolutely continuous part of $Q_n$ relative to $P_n$ converges in distribution under $P_n$ to $L$, with $L>0$ almost surely and $\mathbb{E}L=1$, then the two sequences are mutually contiguous. This includes the usual case $L_n=d{Q}_n/d{P}_n$. The mean-one condition ensures uniform integrability of $L_n$ and vanishing mass of any singular part; positivity ensures the reverse implication.

Take $P_n=P_{\theta }^n$ and $Q_n=P_{\theta +h/\sqrt{n}}^n$. Their likelihood ratio on the support of $P_n$ satisfies, by local asymptotic normality and the continuous mapping theorem,This limit is positive and $\mathbb{E}L=e^{-v/2}{e}^{v/2}=1$, including $v=0$. The conditions of Le Cam's first lemma therefore hold, giving mutual contiguity of the product sampling laws. The products, rather than the one-observation laws, are the relevant sequences.

For any $\epsilon >0$, let $A_n=\{\Vert {\hat{\theta }}_n-\theta \Vert >\epsilon \}$. Consistency under $P_{\theta }^n$ makes $P_{\theta }^n(A_n)\to 0$, and contiguity gives $Q_n(A_n)\to 0$. Thus the estimator remains consistent for $\theta$ under each fixed local alternative. Since $\Vert \theta +h/\sqrt{n}-\theta \Vert \to 0$, it is also consistent for the moving parameter in the sense that $\Vert {\hat{\theta }}_n-(\theta +h/\sqrt{n})\Vert \to 0$ in probability under $Q_n$.

Let $W_n$ be the interpolation in part (a). The operationis continuous, since $|I(f)-I(g)|\le \Vert f-g{\Vert }_{\infty }$. By the Donsker invariance principle and the continuous mapping theorem,The exact trapezoidal integral of the linear interpolation isTherefore the statistic in question differs from $I(W_n)$ by $S_n/(2n^{3/2})$. Using independence, zero means, and unit variances givesThis error tends to zero in $L^2$, hence in probability. The Slutsky theorem now proves the integrated random-walk limit:The limiting law can also be made explicit. The time integral is a Gaussian random variable, as a mean-square limit of linear combinations of a Gaussian process. It is centered, and the covariance identity $\mathbb{E}(B_s{B}_t)=\min (s,t)$ givesThus the terminal value of integrated Brownian motion here has law $N(0,1/3)$.

Use the Donsker invariance principle: for independent and identically distributed random variables of mean zero and variance one, the linearly interpolated processes $W_n$ with $W_n(k/n)=S_k/\sqrt{n}$ converge in distribution to standard Brownian motion in $C[0,1]$ with the uniform topology. No moment beyond the finite second moment is required.

The maximum functional $F(f)={\max }_{0\le t\le 1}f(t)$ is continuous, since $|F(f)-F(g)|\le \Vert f-g{\Vert }_{\infty }$. A linear function on each interpolation interval has its maximum at an endpoint, so $F(W_n)=M_n/\sqrt{n}$. The continuous mapping theorem therefore givesBy the Brownian reflection principle, the limiting random variable has the distribution of $|Z|$, $Z\sim N(0,1)$. Its distribution function is continuous and has no atom at any $x\ge 0$, so convergence in distribution permits passage to these tail probabilities. HenceAt $x=0$ the left side is exactly one for every $n$, because $S_0=0$ is included in the maximum, and the right side is also one.

The almost-sure bound gives ${\tilde{\theta }}_n-{\hat{\theta }}_n=o_{\mathbb{P}}(n^{-1/2})$, so Slutsky theorem transfers both consistency and the asymptotic normal law to ${\tilde{\theta }}_n$. The continuous mapping theorem and delta method then giveIf ${\psi }^{\prime }({\theta }_0)=0$, the latter is the Dirac measure at zero, conventionally a degenerate normal distribution; a nondegenerate second-order limit would need a different normalization and extra smoothness.

Under the null model, $\pi =1$, which is a boundary point of the parameter space. Standard interior-point asymptotic normality of the maximum-likelihood estimator therefore need not apply.

Indeed, put $a=1-e^{-\lambda }$. Under $\pi =1$, $N_{+}\sim \mathrm{Binomial}(n,a)$. For the unconstrained estimator $\tilde{\pi }=N_{+}/(na)$, the central limit theorem givesSince $\hat{\pi }=\min \{1,\tilde{\pi }\}$, the continuous mapping theorem yieldsThis distribution has probability $1/2$ at zero and a continuous normal half-density on the negative half-line, so it is not normal.

The strong law of large numbers gives $S_n/n=\overline{X}\to 1/\theta$ almost surely, and hence in probability. Thereforeby the continuous mapping theorem. Thus the Bayes estimator has consistency.

It differs from the maximum-likelihood estimator bySince $S_n/n\to 1/\theta$, the numerator is $O_p(n)$ and the denominator is $O_p(n^2)$, so this difference is $O_p(n^{-1})=o_p(n^{-1/2})$. The limit in part a and Slutsky theorem now give

The strong law of large numbers givesA continuous one-to-one real function is strictly monotone and has a continuous inverse on its image. Hencewhenever ${\hat{\mu }}_n$ lies in that image; an arbitrary extension outside the image handles the remaining finite-sample values. The continuous mapping theorem gives ${\hat{\theta }}_n\to {\theta }_0$ almost surely, so this is a consistent estimator.

PutThe log-likelihood, apart from an additive constant, isOver the full parameter space its maximum-likelihood estimates are $(\hat{\mu },{\hat{\sigma }}^2)=(\overline{X},S/n)$. Under $H_0$, maximizing over the nuisance parameter $\mu$ gives $({\hat{\mu }}_0,{\sigma }_0^2)=(\overline{X},1)$. Thus the generalized likelihood-ratio test statistic, in the convention $2({\sup }_{\Theta }{\ell }_n-{\sup }_{{\Theta }_0}{\ell }_n)$, is

Under $H_0$, the normal-sample decomposition givesWriting $Z_n=(S-(n-1))/\sqrt{2(n-1)}$, the central limit theorem for a sum of squared independent standard normal variables yields $Z_n\stackrel{d}{\to }N(0,1)$. Hence Slutsky theorem givesand in particular ${\hat{\sigma }}^2\stackrel{P}{\to }1$.

For $h(x)=x-1-\log x$, Taylor theorem at $x=1$ givesConsequentlyby the continuous mapping theorem and Slutsky's theorem. Since the square of a standard normal variable has the chi-squared distribution with one degree of freedom,

The Poisson central limit theorem, or the ordinary central limit theorem applied to the $X_i$, gives $Y_n\stackrel{d}{\to }Y$ for a standard normal random variable $Y$. The negative-part map $x\mapsto x^{-}:=\max (-x,0)$ is continuous, so the continuous mapping theorem gives

WriteExpanding $R_i^2$ gives the leading term ${\epsilon }_i^2{\xi }_i^2$ and terms of the forms treated in parts b and c, together with their versions obtained by interchanging $(\epsilon ,F)$ and $(\xi ,G)$. For example, the pure error terms are ${\epsilon }_i^2{G}_i^2$, ${\xi }_i^2{F}_i^2$, and $F_i^2{G}_i^2$, and each cross term is controlled by Cauchy-Schwarz from these. HenceAssuming this limit is positive, the continuous mapping theorem yields ${\tau }_D\stackrel{p}{\to }\{\mathbb{E}({\epsilon }_1^2{\xi }_1^2){\}}^{1/2}$. Combining this with the assumed convergence in distribution of $\sqrt{n}{\tau }_N$ and applying the Slutsky theorem givesThis is the studentization of the generalized covariance measure statistic.

Let $g:M^{\prime }\to \mathbb{R}$ be bounded and continuous. The composition $h=g\circ f$ is bounded and measurable, and every discontinuity point of $h$ is a discontinuity point of $f$. ThusThe Portmanteau theorem includes the null-discontinuity criterion: if $X_n\stackrel{d}{\to }X$ and a bounded measurable function is continuous at $X$ almost surely, then its expectations converge. HenceSince this holds for every bounded continuous $g$, it proves the continuous mapping theorem conclusion $f(X_n)\stackrel{d}{\to }f(X)$.

Use the test statisticUnder the null hypothesis, the Hilbert-space central limit theorem gives $\sqrt{n}{\overline{X}}_n\stackrel{d}{\to }G$, where $G$ is centered Gaussian with covariance $C_X$. If $C_X{\varphi }_j={\lambda }_j{\varphi }_j$, its Karhunen–Loève expansion and the continuous mapping theorem givefor independent $Z_j\sim N(0,1)$. Reject for $T_n$ above the $(1-\alpha )$ quantile of this weighted chi-squared law; replacing the ${\lambda }_j$ by empirical covariance eigenvalues gives a plug-in estimator of the critical value.

Under every fixed alternative $\mu \ne 0$, the weak law of large numbers gives ${\overline{X}}_n\stackrel{p}{\to }\mu$, so $T_n/n\stackrel{p}{\to }\Vert \mu {\Vert }^2$ and the test is consistent. Under a local alternative ${\mu }_n=n^{-1/2}\delta$, the limit is $\Vert G+\delta {\Vert }^2$, which describes its local power.

Independence makes the joint law of $(X_n,Y_n)$ the product of its marginal laws. Weak convergence of both marginals implies convergence of these product measures to the product law of independent copies $(X,Y)$. The addition map is continuous, so the continuous mapping theorem gives $X_n+Y_n\stackrel{d}{\to }X+Y$.

Reversing the finite summations givesThis is a Riemann sum for the continuous functional $f\mapsto {\int }_0^{1}f(t)dt$ evaluated at the interpolated random walk; the interpolation error tends to zero in probability. The continuous mapping theorem and part (b) yield