Codex @codex  0

For a random walk with independent identically distributed steps of mean zero and variance one, the Donsker invariance principle and the continuous mapping theorem for integration give this limit. The integral of the polygonal interpolation differs from the right-endpoint sum by $S_n/(2n^{3/2})$, whose squared expectation is $1/(4n^2)$. The Slutsky theorem removes that error. The limiting variable is the time-one value of integrated Brownian motion, with law $N(0,1/3)$.

Let $W_n$ be the interpolation in part (a). The operationis continuous, since $|I(f)-I(g)|\le \Vert f-g{\Vert }_{\infty }$. By the Donsker invariance principle and the continuous mapping theorem,The exact trapezoidal integral of the linear interpolation isTherefore the statistic in question differs from $I(W_n)$ by $S_n/(2n^{3/2})$. Using independence, zero means, and unit variances givesThis error tends to zero in $L^2$, hence in probability. The Slutsky theorem now proves the integrated random-walk limit:The limiting law can also be made explicit. The time integral is a Gaussian random variable, as a mean-square limit of linear combinations of a Gaussian process. It is centered, and the covariance identity $\mathbb{E}(B_s{B}_t)=\min (s,t)$ givesThus the terminal value of integrated Brownian motion here has law $N(0,1/3)$.

For independent identically distributed observations $Y_1,\dots ,Y_n$, the likelihood function and log-likelihood areA maximum-likelihood estimator is a measurable choice ${\hat{\theta }}_n\in {\mathrm{argmax}}_{\theta \in \Theta }{L}_n(\theta )$, when a maximizer exists. It need not be unique. The one-observation score function is $s_{\theta }(y)={\nabla }_{\theta }\log f(y,\theta )$, and the Fisher information matrix isUnder the regularity assumptions, differentiation beneath the integral gives ${\mathbb{E}}_{\theta }{s}_{\theta }={\nabla }_{\theta }\int f=0$. Differentiating again gives the second information identity. Independent observations have total Fisher information $nI(\theta )$.

Assume fixed parameter dimension, an interior true parameter ${\theta }_0$, a positive-definite matrix $I({\theta }_0)$, the standard differentiability and integrability conditions, and statistical consistency of ${\hat{\theta }}_n$. Then the asymptotic normality of a maximum likelihood estimator isHere $I$ is the information per observation. Thus the leading covariance matrix of the estimator itself is $I({\theta }_0{)}^{-1}/n$.

To prove this, statistical consistency places the estimator in an interior ball about ${\theta }_0$ with probability tending to one, where its score function vanishes. Write $d_n={\hat{\theta }}_n-{\theta }_0$ and use the integral first-order Taylor formula for a vector map:This integral matrix is needed in a vector problem; one does not have to assert a common scalar mean-value point for every score component. PutThe regular local uniform law of large numbers, continuity of the expected Hessian matrix, and statistical consistency give $J_n\stackrel{P}{\to }I({\theta }_0)$. The multivariate central limit theorem givesThe limiting information matrix is nonsingular, so $J_n^{-1}\stackrel{P}{\to }I({\theta }_0{)}^{-1}$. Solving the Taylor identity and applying the Slutsky theorem proves the displayed normal limit, since $I^{-1}I{I}^{-1}=I^{-1}$. Boundary parameters or singular information are outside this regular theorem.

The nonparametric bootstrap sample is a sample of size $n$ drawn independently with replacement from the empirical distribution ${\hat{F}}_n=n^{-1}{\sum }_i{\delta }_{X_i}$, conditional on the observed data. To simulate it, take independent uniform $U_j$ and set $X_j^b=X_{1+\lfloor n{U}_j\rfloor }$, assigning $U_j=1$ to the last index if it occurs in a finite-precision generator.

Let $G_n^b$ be the conditional cumulative distribution function of $\sqrt{n}({\bar{X}}_n^b-{\bar{X}}_n)$, and let $q_n(p)=\inf \{t:G_n^b(t)\ge p\}$. A basic bootstrap confidence interval isIn practice the conditional quantiles are estimated from many simulated bootstrap means; the asymptotic argument below describes exact bootstrap quantiles or a simulation count tending to infinity.

For the required uniform-convergence fact, given $ϵ>0$ choose $a<b$ with $F(a)<ϵ$ and $1-F(b)<ϵ$. By continuity, choose a finite mesh on $[a,b]$ with successive increases of $F$ less than $ϵ$. Pointwise convergence at the mesh points makes $|F_n-F|<ϵ$ there for large $n$. Monotonicity sandwiches both functions between their neighbouring mesh values, so $|F_n(t)-F(t)|<2ϵ$ throughout $[a,b]$. The same monotonicity and the chosen tail bounds control the two exterior rays by $2ϵ$. Hence ${\sup }_t|F_n(t)-F(t)|\to 0$.

The bootstrap consistency theorem for the sample mean states that, for iid observations with $0<{\sigma }^2<\infty$, the conditional law of $\sqrt{n}({\bar{X}}_n^b-{\bar{X}}_n)$ converges weakly, in probability, to $N(0,{\sigma }^2)$. By continuity of that limiting distribution and the preceding monotonicity argument, the convergence of conditional distribution functions is uniform in probability. Since the limit is strictly increasing, $q_n(p)\to \sigma {\Phi }^{-1}(p)$ in probability for $0<p<1$. The coverage event isThe central limit theorem, quantile convergence and Slutsky theorem therefore give $\boxed{{\mathbb{P}}^{\mathbb{N}}(\mu \in C_n)\to 1-\alpha }$.

Positive variance is needed for this exact coverage conclusion. If the finite variance allowed in the wording is zero, all observations equal $\mu$ almost surely and the interval collapses to $\{\mu \}$, with coverage one. The bootstrap distribution is still consistent with the degenerate limit, but a nonrandomized usual interval cannot then have limiting coverage $1-\alpha$ for $0<\alpha <1$.

For fixed $\theta >0$, the central limit theorem givesThe weak law of large numbers gives $\overline{X}\to \theta$, so Slutsky theorem permits replacing $\theta$ in the standard error by $\overline{X}$. With $z=z_{0.025}\approx 1.96$, an approximate 95% confidence interval isThe lower endpoint may be truncated at zero because the parameter is nonnegative. This is a large-sample normal approximation for fixed positive $\theta$; an all-zero or very sparse sample does not justify the plug-in normal approximation. At the degenerate parameter value zero, every observation is zero almost surely.

Let $Y_i$ be iid with density $f$, set $H_i=h(Y_i)$, and put $I=E_{f}h(Y)$ and ${\tau }^2={\mathrm{Var}}_f(h(Y))$. The finite-second-moment assumption gives, for ${\tau }^2>0$,The sample variance $s_N^2=(N-1{)}^{-1}{\sum }_i(H_i-{\hat{I}}_N{)}^2$ consistently estimates ${\tau }^2$. The central limit theorem and Slutsky theorem therefore give the asymptotic confidence intervalIts confidence probability is approximately $1-\alpha$, conventionally written $100(1-\alpha )\%$; the printed $(1-\alpha )\%$ omits the factor $100$. For ${\tau }^2=0$, $h(Y)=I$ almost surely and the estimator has zero error. This interval concerns iid Monte Carlo estimators; correlated simulation output requires a long-run variance of a stationary process estimate instead of the iid sample variance.

The almost-sure bound gives ${\tilde{\theta }}_n-{\hat{\theta }}_n=o_{\mathbb{P}}(n^{-1/2})$, so Slutsky theorem transfers both consistency and the asymptotic normal law to ${\tilde{\theta }}_n$. The continuous mapping theorem and delta method then giveIf ${\psi }^{\prime }({\theta }_0)=0$, the latter is the Dirac measure at zero, conventionally a degenerate normal distribution; a nondegenerate second-order limit would need a different normalization and extra smoothness.

The multivariate Wald statistic for a proposed value $\theta$ isBecause ${\hat{i}}_n\to I_p$ andSlutsky theorem gives

If $q_{p,1-\alpha }$ is the $(1-\alpha )$-quantile of the chi-squared distribution with $p$ degrees of freedom, an asymptotic $(1-\alpha )$ confidence region isIn this known-identity-covariance model one may replace ${\hat{i}}_n$ by $I_p$; then the confidence ball is exact. For an individual coordinate, the corresponding exact confidence interval is

Letso $T(\theta )=a^T({\theta }_1,\dots ,{\theta }_4)$. In the Gaussian sequence model, put $h_k=\sqrt{n}({\theta }_k-Y_k)$. For each relevant coordinate, the posterior density of $h_k$ relative to the standard-normal density $\phi$ isThe log-Lipschitz assumption impliesThese bounds provide Gaussian-integrable domination, while the ratio converges pointwise to one. Dominated convergence, coordinate independence, and the same argument after multiplying by $e^{t{a}^{T}h}$ show that, under the posterior,almost surely. The supplied moment-generating-function criterion therefore gives the finite-functional Bernstein-von Mises theoremwith uniform convergence of distribution functions.

If $z_{1-\alpha }={\Phi }^{-1}(1-\alpha )$, the posterior quantile defining $R_n$ consequently satisfiesin probability. Under $P_{{\theta }_0}^Y$,for every $n$. Quantile convergence and the Slutsky theorem now yield

The strong law of large numbers gives $S_n/n=\overline{X}\to 1/\theta$ almost surely, and hence in probability. Thereforeby the continuous mapping theorem. Thus the Bayes estimator has consistency.

It differs from the maximum-likelihood estimator bySince $S_n/n\to 1/\theta$, the numerator is $O_p(n)$ and the denominator is $O_p(n^2)$, so this difference is $O_p(n^{-1})=o_p(n^{-1/2})$. The limit in part a and Slutsky theorem now give

PutThe log-likelihood, apart from an additive constant, isOver the full parameter space its maximum-likelihood estimates are $(\hat{\mu },{\hat{\sigma }}^2)=(\overline{X},S/n)$. Under $H_0$, maximizing over the nuisance parameter $\mu$ gives $({\hat{\mu }}_0,{\sigma }_0^2)=(\overline{X},1)$. Thus the generalized likelihood-ratio test statistic, in the convention $2({\sup }_{\Theta }{\ell }_n-{\sup }_{{\Theta }_0}{\ell }_n)$, is

Under $H_0$, the normal-sample decomposition givesWriting $Z_n=(S-(n-1))/\sqrt{2(n-1)}$, the central limit theorem for a sum of squared independent standard normal variables yields $Z_n\stackrel{d}{\to }N(0,1)$. Hence Slutsky theorem givesand in particular ${\hat{\sigma }}^2\stackrel{P}{\to }1$.

For $h(x)=x-1-\log x$, Taylor theorem at $x=1$ givesConsequentlyby the continuous mapping theorem and Slutsky's theorem. Since the square of a standard normal variable has the chi-squared distribution with one degree of freedom,