Box-kernel bias of a piecewise constant density Created 2026-10-07 Updated 2026-11-05
For a density with finitely many jumps at separated points and the unit-width box kernel, take smaller than all consecutive jump spacings. The smoothing bias is supported in disjoint length- neighbourhoods of the jumps. At each jump its two sides form triangles of height , and exact integration givesCombined with the integrated variance of a kernel density estimator, this gives expected error at .
For an equally spaced design with independent errors of variance at most and a regression function with Lipschitz constant , the unit-width box kernel estimator averages the observations in the window. Its bias is at most , and its variance is at most . The window occupancy for an equally spaced regression design gives, whenever ,In the usual range , this is uniformly up to the boundary. For arbitrarily large bandwidth, the stochastic error cannot continue to decrease as : once all observations are included the estimator is simply their mean.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 33 2 Solution Created 2026-10-03 Updated 2026-10-14
For a normalized kernel for density estimation , with , the kernel density estimator isIt replaces each atom of the empirical measure by a translated, scaled copy of : averaging these small probability-density bumps gives a smooth approximation to the sampling density. Integral one is part of the probability-kernel convention; without it, the estimator generally targets .
The printed bandwidth assumption is insufficient for the claimed normal limit. The additional condition is needed at points with positive density. For a counterexample, take a normal distribution density , , the unit-width box kernel and . Then , but the probability that any observation lies in the window is at most . On the complementary event the estimator is zero, and . The limit in probability is therefore zero, whereas the stated normal limit has positive variance.
We now prove the intended pointwise central limit theorem for a kernel density estimator under , and . The first condition also follows from the last one. A mean value theorem bound, with , gives the bias of a kernel density estimatorThus the bias multiplied by tends to zero.
The Lindeberg-Feller central limit theorem says that independent centered variables in each row of a triangular array have sum converging in distribution to if the sum of their variances tends to and, for every , the sum of their truncated second moments tends to zero. Put and . The independence hypothesis holds within each row. A change of variable givesContinuity of at and the compact support and boundedness of justify the limit. Also , so the Lindeberg condition is eventually identically zero for every . The sum is exactly . Combining the centered limit with the negligible bias by the Slutsky theorem provesIf , the centered variance tends to zero, and the Chebyshev inequality proves the corresponding degenerate limit directly.
For , let be the supplied -quantile of this limiting normal distribution. Inverting the limiting central probability gives the confidence intervalContinuity of the limiting distribution gives asymptotic coverage . Equivalently, its half-width is under the stipulated oracle quantiles. A degenerate limit at does not by itself justify this nominal-coverage normal interval.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 33 4 Solution Created 2026-10-03 Updated 2026-10-14
The Nadaraya–Watson estimator isFor the unit-width box kernel, it averages the responses with design points in . Define it to be zero if that window contains no observations; the supplied small-denominator bound must refer to a defined estimator.
Write , and . Set . On ,This is the absolute error bound for a ratio estimator with a controlled denominator; it does not require the numerator and denominator to be independent.
The conditional expectation definition of the regression function givesAt this interior point, the Taylor theorem expansions are and . Multiplying and using the symmetric interval removes the odd linear term. More precisely,This cancellation underlies the interior absolute-error rate of the Nadaraya-Watson estimator.
For the variance, independence between observations givesConditional on , the second moment is . The bounded conditional variance, local boundedness of , and boundedness of consequently make the expectation at most . Thus . By the Cauchy-Schwarz inequality,The auxiliary result supplied in the question makes the expected error on equal to . Combining both events, with , provesBoth the squared-bandwidth bias and the inverse-square-root window-count error have this order. The supplied separate consistency result for the design-density estimator is not needed once the stronger small-denominator expectation bound is used.
Past exam of the mathematics course of the University of Cambridge 2013 iii Paper 33 2 Solution Created 2026-10-03 Updated 2026-10-07
For the unit-width box kernel, the convolution and kernel density estimator areandIn particular . The scaling gives . Even if is not square-integrable, Young's convolution inequality gives because and .
Using independence to eliminate the cross terms in the centred estimator, and Tonelli theorem to integrate the nonnegative variance, gives the exact integrated variance of a kernel density estimator:The Cauchy-Schwarz inequality now yieldsThis constant comes from the unit norm of the unscaled unit-width box kernel.
For the piecewise constant probability density function, integrability forces the constants on both unbounded outer intervals to be zero. Thus is bounded, has compact support, and has finitely many jumps. Let denote the jump at . For smaller than the least gap between consecutive breakpoints, the smoothing regions do not overlap. Outside these regions the convolution equals .
Within a region, put . For , the bias is ; for , it is . Endpoint values do not affect an norm. Integrating these two triangular errors givesThis is the box-kernel bias of a piecewise constant density. By the triangle inequality, with ,Balancing this bias-variance tradeoff with gives, for all sufficiently large ,
Past exam of the mathematics course of the University of Cambridge 2013 iii Paper 33 4 Solution Created 2026-10-03 Updated 2026-10-07
For the unit-width box kernel, the order-zero local polynomial estimator minimizes, over constants ,A common factor such as in these weights does not change the minimizer. Put and . If , differentiating the quadratic gives its unique minimizer:The last expression is the Nadaraya–Watson estimator. Every is within of some design point . Since gives , that point is in the closed kernel window. Thus and the equality holds throughout the domain, including and .
For the error bound, first take the usual bandwidth range . The clipped window has length at least . Any closed interval of length in contains at least points of the design . Consequentlywhere the last inequality uses . This is a window occupancy for an equally spaced regression design bound that remains valid at the boundary.
Let , the Lipschitz constant supplied by the bounded-derivative assumption. The bias is bounded byIndependence and the variance bound giveBy the Cauchy-Schwarz inequality and the triangle inequality,Thus works uniformly in and the design size in this bandwidth range. The same clipped-length proof in fact works for .
There is a genuine omission in the unrestricted formulation: an upper restriction on bandwidth is necessary for the displayed variance scale. Take , , and . The window contains the sole observation for every , and holds. Its mean absolute error is , whereas the proposed variance term is . No fixed can satisfy this for unbounded .
A valid statement for every satisfying uses . The clipped window has length at least , so , while the preceding coverage argument gives . If , the first bound gives ; if , the second does. Therefore the same bias and variance calculation proves the unrestricted correctionThis mean absolute error of local constant regression bound reduces to the requested rate for ordinary small bandwidths and saturates its stochastic term when the window covers the whole design.