Integrated random-walk limit Created 2026-10-07 Updated 2026-11-05
For a random walk with independent identically distributed steps of mean zero and variance one, the Donsker invariance principle and the continuous mapping theorem for integration give this limit. The integral of the polygonal interpolation differs from the right-endpoint sum by , whose squared expectation is . The Slutsky theorem removes that error. The limiting variable is the time-one value of integrated Brownian motion, with law .
Marginal weak convergence does not control sums Created 2026-10-14 Updated 2026-11-05
Let take values with equal probabilities. Set for every , and for even , for odd . Both marginal laws are constant, but alternates between the law of and a point mass at zero. Thus separate weak convergence of random variables does not determine a joint limit or a sum limit. The obstruction disappears when one marginal limit is constant, as in the Slutsky theorem.
Write the constant limit as . First, convergence in distribution to a constant implies convergence in probability: the Portmanteau theorem applied to the closed set gives
Now compare characteristic functions without imposing independence. For fixed ,
The second bound uses . By convergence in probability, the limiting upper bound is ; then let . Meanwhile has weak convergence of random variables to , so . Thus the characteristic function of the sum tends to , the characteristic function of , which is continuous at zero. The Lévy continuity theorem proves
This is the additive case of the Slutsky theorem, with its proof supplied. The constant limiting marginal removes the dependence obstruction in part (b).
For fixed , use the Lyapunov condition for the independent centered epoch counts . Their total variance is .
Write . Differentiating the geometric distribution probability generating function gives . Since , the sum of the centered absolute third moments is . Consequently
The Lindeberg-Feller central limit theorem applies through the Lyapunov condition, giving . Now , so the Slutsky theorem proves the asymptotic normality of the Watterson estimator:
The limiting variance is . When , every count vanishes and the limiting law is the point mass at zero.
Write and for the individual and total score functions. Under the usual regularity assumptions, and , the nonsingular per-observation Fisher information matrix. The central limit theorem and weak law of large numbers give
Rearranging the stipulated score expansion, including its remainder, gives
Slutsky theorem therefore proves the asymptotic normality of a maximum likelihood estimator:
A consistent plug-in Fisher information matrix yields the Wald statistic . Under the null hypothesis, , so a test of asymptotic size rejects when exceeds the quantile of that chi-squared distribution. A consistent observed Fisher information matrix gives the same limiting test.
With an -dimensional parameter of interest and nuisance parameter , partition accordingly. The marginal asymptotic covariance matrix of is the block of , whose inverse is the efficient information
Thus the nuisance-adjusted Wald statistic is , with limiting null distribution . Using alone is generally incorrect when the nuisance parameter is estimated; it is valid when the cross-information vanishes.
For the Inverse Gaussian distribution in this problem, expand the exponent to obtain the log likelihood, up to terms independent of both parameters,
Its score functions are
For each positive , the first score changes sign from positive to negative at , so this is the global maximizing mean parameter of an exponential family. Substituting it in the second score gives
The reciprocal function is strictly convex, so the denominator is positive for a nonconstant sample. The profile log likelihood is strictly concave in and tends to minus infinity at both endpoints, proving this is a maximum. Under continuous model sampling, a sample of size at least two is nonconstant almost surely. For a constant sample, including a one-observation sample, no finite joint maximum-likelihood estimator exists: the likelihood increases without bound as at .
For one observation, differentiating again and using gives
This parameter orthogonality proves that estimating does not reduce the efficient information for . The Wald statistic has the same algebraic form in the two cases:
Using information evaluated at instead is another first-order equivalent convention.
The printed claim of coincidence needs a first-order interpretation. Known and unknown need not give numerically identical Wald statistics, because their maximum-likelihood estimators differ. With known, direct maximization gives
For example, observations and known give and . At , the displayed Wald convention gives and , respectively. Thus exact equality is false. Under the null hypothesis, however, , hence . Their centered, standardized estimates differ by , so their Wald statistics differ by . This is the valid asymptotic coincidence supplied by inverse Gaussian shape estimation and nuisance orthogonality.
For a nondegenerate distribution function , membership in its maximum domain of attraction means that there exist such that
at every continuity point of . A max-stable distribution satisfies: for each positive integer , there exist such that . Thus a maximum of independent copies has the same probability distribution after a change of location and scale.
Suppose first that . For fixed , the distribution functions have the two nondegenerate limits
The given convergence-of-types result supplies constants such that . Equality extends from continuity points to all points by right continuity. Substitute to obtain , which proves max-stability with . Conversely, if is max-stable, take and its max-stability constants for . Then exactly for every , proving . Therefore the maximum domain of attraction is nonempty exactly when is max-stable.
For the particular logarithmic tail, complete the distribution function by taking for . Let be the unique solution of , and take . Equivalently , including with . For each fixed , eventually , and
Here follows directly from its defining equation. Since as , it follows that . For , the probability is zero. Consequently
This is the shape-one Fréchet distribution, and the calculation proves the limit at every real without needing a regular variation theorem.
The function is strictly increasing on . For every integer , , so . The defining equation now gives . Taking logarithms yields , whose right side is positive, so
Multiplying by and squeezing gives . Slutsky theorem applied to the normalized sample maximum therefore gives the simpler logarithmic-tail Fréchet normalization
Assume an identifiable smooth parametric model, an interior true parameter, a consistent interior maximum-likelihood estimator, interchange of differentiation and integration, a uniform local law of large numbers for the Hessian, and nonsingular finite Fisher information. For one observation define the score function and
The score function has expectation zero by differentiating the normalization of the density. Its sample sum satisfies by the vector central limit theorem. The score function equation and Taylor's formula give
The bracket divided by converges in probability to by consistency and the assumed uniform law. Solving the equation and applying Slutsky theorem yields
The integral Hessian avoids incorrectly assuming a single intermediate point for a vector-valued Taylor formula.
Set , and . The fourth-moment assumption makes finite, so the variance of a sample mean is . Assume , as required for leave-one-out estimation.
The Jackknife variance estimator recomputes the original statistic on each sample with one observation removed. Here
By the definition of the Jackknife variance estimator,
For completeness, expanding the centered sum gives
Consequently . The jackknife variance estimator is unbiased for this statistic. This calculation uses only the finite fourth moment, not normality of the original observations.
Given the observed data, put and . The empirical distribution assigns mass to each observed , including repeated values with their multiplicities. A bootstrap sample consists of independent draws from this distribution, with replacement. The nonparametric bootstrap variance estimate is the conditional variance
Independence of the bootstrap draws gives the exact conditional bootstrap variance of a sample mean:
Thus its finite-sample expectation is , in contrast to the unbiased jackknife estimate. This is the jackknife and bootstrap variance of a sample second moment scaling difference.
To approximate it by Monte Carlo simulation, independently generate bootstrap samples. For replicate , choose independent uniform indices and compute . With , use
For this is conditionally unbiased for the bootstrap variance and converges to it as . Uniform indices can be obtained by for uniforms on .
Let be the conditional bootstrap distribution of the statistic. A nominal level percentile bootstrap confidence interval is
Reuse the bootstrap replicates, sort them as , and take the order statistics with ranks and as empirical quantiles. These are quantiles of the statistic itself, rather than reflected quantiles of its centered error. Increasing reduces simulation error; it does not turn an approximate confidence interval into an exact finite-sample one.
An analytic alternative follows from the central limit theorem. If , then , and the sample variance converges to by the strong law of large numbers. The Slutsky theorem therefore gives the large-sample normal approximation interval
where is the standard normal quantile. Its lower endpoint may be truncated to zero because the parameter is nonnegative. If , almost surely and the statistic identifies the parameter exactly; the interval degenerates to that value. The nondegenerate normal interval is asymptotic, not an exact Student interval under arbitrary .
Start the planar Brownian motion at and stop at . The complex exponential is holomorphic with nonzero derivative. By conformal invariance of planar Brownian motion, the process , run on its conformal Brownian clock
is planar Brownian motion started at , until its first exit from the unit disk. The clock is strictly increasing before , and almost surely. It is legitimate to use the exponential although it is not globally injective: the Brownian coordinate brackets depend locally on the nonzero derivative, and the clock construction gives the stopped planar Brownian law. Before , ; at it equals one. Hence the disk exit time in the new clock is exactly .
The continuous lifted argument of this path, starting from angle zero, is . Its total argument at exit is therefore , which has distribution by part (b). If the winding count records only completed integer turns, it differs from by a remainder with . If fractional turns are retained, that remainder is zero. In either convention, since ,
The first term has distribution for every , and the second tends to zero deterministically in absolute value as . Symmetry of the Cauchy distribution removes the minus sign, so Slutsky theorem gives the small-radius Brownian winding law:
Let for independent and identically distributed random variables with distribution function . Then . Membership in the maximum domain of attraction of a nondegenerate distribution function means that some deterministic and satisfy
or equivalently at every continuity point of .
Under the stated convergence, put , , and . The Slutsky theorem says that combining a sequence with convergence in distribution with sequences converging in probability to constants preserves the corresponding sum, product and ratio limits, provided a limiting denominator is nonzero. Applying it here gives
The limiting distribution function is , continuous for every real because is continuous and . Consequently,
For a normalized kernel for density estimation , with , the kernel density estimator is
It replaces each atom of the empirical measure by a translated, scaled copy of : averaging these small probability-density bumps gives a smooth approximation to the sampling density. Integral one is part of the probability-kernel convention; without it, the estimator generally targets .
The printed bandwidth assumption is insufficient for the claimed normal limit. The additional condition is needed at points with positive density. For a counterexample, take a normal distribution density , , the unit-width box kernel and . Then , but the probability that any observation lies in the window is at most . On the complementary event the estimator is zero, and . The limit in probability is therefore zero, whereas the stated normal limit has positive variance.
We now prove the intended pointwise central limit theorem for a kernel density estimator under , and . The first condition also follows from the last one. A mean value theorem bound, with , gives the bias of a kernel density estimator
Thus the bias multiplied by tends to zero.
The Lindeberg-Feller central limit theorem says that independent centered variables in each row of a triangular array have sum converging in distribution to if the sum of their variances tends to and, for every , the sum of their truncated second moments tends to zero. Put and . The independence hypothesis holds within each row. A change of variable gives
Continuity of at and the compact support and boundedness of justify the limit. Also , so the Lindeberg condition is eventually identically zero for every . The sum is exactly . Combining the centered limit with the negligible bias by the Slutsky theorem proves
If , the centered variance tends to zero, and the Chebyshev inequality proves the corresponding degenerate limit directly.
For , let be the supplied -quantile of this limiting normal distribution. Inverting the limiting central probability gives the confidence interval
Continuity of the limiting distribution gives asymptotic coverage . Equivalently, its half-width is under the stipulated oracle quantiles. A degenerate limit at does not by itself justify this nominal-coverage normal interval.
A real functional is Hadamard differentiable at if there exists a continuous linear functional such that, whenever , and in the normed vector space,
If is defined only on a subset, require to lie in its domain. The variation of the directions is essential: convergence along each fixed direction alone is a weaker differentiability condition. Tangential Hadamard differentiability restricts the limit directions to a specified subspace; here the derivative is defined on all of .
We prove the functional delta method using compact sets, so do not need an unjustified application of the Skorokhod representation theorem on an arbitrary nonseparable normed vector space. As usual for function-space convergence in distribution, the limiting random element is tight, and the real statistics are measurable, or convergence is understood through outer expectations. These are automatic under the usual separable complete random-element formulation; the empirical-process formulation used below has a tight separably supported limit.
For put
Hadamard differentiability says for every convergent sequence of directions, since continuity gives . It implies a Hadamard remainder near a compact set: for every compact and there are and such that
To prove it, suppose it failed. Choose and at distance less than from with . Choose with . A subsequence of converges by compactness, so the corresponding converge too. The defining derivative limit contradicts the lower bound on .
Now let and . Given , tightness of a probability measure provides a compact with . For the open -neighbourhood , the Portmanteau theorem gives
On the preceding uniform remainder is less than for all sufficiently large . Consequently . Since is arbitrary, this remainder tends to zero in probability. We have proved the actual expansion
Continuity of and the continuous mapping theorem give . The vanishing remainder and Slutsky theorem therefore yield
This derivation uses the derivative definition, rather than assuming a uniform Fréchet remainder over every bounded set. If outer expectations are needed, the same compact-neighbourhood argument uses outer probabilities and gives the corresponding empirical-process functional delta method.
For the statistical application, use the sample's empirical distribution function and the plug-in estimator
Here the printed denotes bounded pointwise functions with the supremum norm, namely , rather than equivalence classes modulo Lebesgue null sets. Assume Hadamard differentiability at the true and the usual measurability of the scalar estimator. The Donsker theorem for empirical distribution functions and the proved functional delta method give
No continuity of is needed for this application. Since the derivative is continuous and linear and the bridge is a centred Gaussian process, is a centred Gaussian real random variable, possibly identically zero.
Finally, a finite real weak limit ensures the required boundedness in probability. Choose so large that . Applying the closed-set part of the Portmanteau theorem to gives
Hence there is such that, for every ,
In particular the upper-tail condition in the question holds. The plug-in estimator has estimation error , even when its limiting variance is zero.
Let be the interpolation in part (a). The operation
is continuous, since . By the Donsker invariance principle and the continuous mapping theorem,
The exact trapezoidal integral of the linear interpolation is
Therefore the statistic in question differs from by . Using independence, zero means, and unit variances gives
This error tends to zero in , hence in probability. The Slutsky theorem now proves the integrated random-walk limit:
The limiting law can also be made explicit. The time integral is a Gaussian random variable, as a mean-square limit of linear combinations of a Gaussian process. It is centered, and the covariance identity gives
Thus the terminal value of integrated Brownian motion here has law .
For independent identically distributed observations , the likelihood function and log-likelihood are
A maximum-likelihood estimator is a measurable choice , when a maximizer exists. It need not be unique. The one-observation score function is , and the Fisher information matrix is
Under the regularity assumptions, differentiation beneath the integral gives . Differentiating again gives the second information identity. Independent observations have total Fisher information .
Assume fixed parameter dimension, an interior true parameter , a positive-definite matrix , the standard differentiability and integrability conditions, and statistical consistency of . Then the asymptotic normality of a maximum likelihood estimator is
Here is the information per observation. Thus the leading covariance matrix of the estimator itself is .
To prove this, statistical consistency places the estimator in an interior ball about with probability tending to one, where its score function vanishes. Write and use the integral first-order Taylor formula for a vector map:
This integral matrix is needed in a vector problem; one does not have to assert a common scalar mean-value point for every score component. Put
The regular local uniform law of large numbers, continuity of the expected Hessian matrix, and statistical consistency give . The multivariate central limit theorem gives
The limiting information matrix is nonsingular, so . Solving the Taylor identity and applying the Slutsky theorem proves the displayed normal limit, since . Boundary parameters or singular information are outside this regular theorem.
The nonparametric bootstrap sample is a sample of size drawn independently with replacement from the empirical distribution , conditional on the observed data. To simulate it, take independent uniform and set , assigning to the last index if it occurs in a finite-precision generator.
Let be the conditional cumulative distribution function of , and let . A basic bootstrap confidence interval is
In practice the conditional quantiles are estimated from many simulated bootstrap means; the asymptotic argument below describes exact bootstrap quantiles or a simulation count tending to infinity.
For the required uniform-convergence fact, given choose with and . By continuity, choose a finite mesh on with successive increases of less than . Pointwise convergence at the mesh points makes there for large . Monotonicity sandwiches both functions between their neighbouring mesh values, so throughout . The same monotonicity and the chosen tail bounds control the two exterior rays by . Hence .
The bootstrap consistency theorem for the sample mean states that, for iid observations with , the conditional law of converges weakly, in probability, to . By continuity of that limiting distribution and the preceding monotonicity argument, the convergence of conditional distribution functions is uniform in probability. Since the limit is strictly increasing, in probability for . The coverage event is
The central limit theorem, quantile convergence and Slutsky theorem therefore give .
Positive variance is needed for this exact coverage conclusion. If the finite variance allowed in the wording is zero, all observations equal almost surely and the interval collapses to , with coverage one. The bootstrap distribution is still consistent with the degenerate limit, but a nonrandomized usual interval cannot then have limiting coverage for .
For fixed , the central limit theorem gives
The weak law of large numbers gives , so Slutsky theorem permits replacing in the standard error by . With , an approximate 95% confidence interval is
The lower endpoint may be truncated at zero because the parameter is nonnegative. This is a large-sample normal approximation for fixed positive ; an all-zero or very sparse sample does not justify the plug-in normal approximation. At the degenerate parameter value zero, every observation is zero almost surely.
Let be iid with density , set , and put and . The finite-second-moment assumption gives, for ,
The sample variance consistently estimates . The central limit theorem and Slutsky theorem therefore give the asymptotic confidence interval
Its confidence probability is approximately , conventionally written ; the printed omits the factor . For , almost surely and the estimator has zero error. This interval concerns iid Monte Carlo estimators; correlated simulation output requires a long-run variance of a stationary process estimate instead of the iid sample variance.
The almost-sure bound gives , so Slutsky theorem transfers both consistency and the asymptotic normal law to . The continuous mapping theorem and delta method then give
If , the latter is the Dirac measure at zero, conventionally a degenerate normal distribution; a nondegenerate second-order limit would need a different normalization and extra smoothness.
The multivariate Wald statistic for a proposed value is
Because and
Slutsky theorem gives
If is the -quantile of the chi-squared distribution with degrees of freedom, an asymptotic confidence region is
In this known-identity-covariance model one may replace by ; then the confidence ball is exact. For an individual coordinate, the corresponding exact confidence interval is
Let
so . In the Gaussian sequence model, put . For each relevant coordinate, the posterior density of relative to the standard-normal density is
The log-Lipschitz assumption implies
These bounds provide Gaussian-integrable domination, while the ratio converges pointwise to one. Dominated convergence, coordinate independence, and the same argument after multiplying by show that, under the posterior,
almost surely. The supplied moment-generating-function criterion therefore gives the finite-functional Bernstein-von Mises theorem
with uniform convergence of distribution functions.
If , the posterior quantile defining consequently satisfies
in probability. Under ,
for every . Quantile convergence and the Slutsky theorem now yield