Kernel for nonparametric regression

A nonnegative regression kernel supplies weights $K((x_i-x)/h)$ in a local weighted least squares fit. Unlike a probability density function, its integral need not equal one, because rescaling every weight does not change the fitted coefficients.

A smoothing bandwidth $h>0$ determines the scale over which a regression kernel averages data, or a kernel for density estimation spreads each observation. It has the same units as the observations. In kernel density estimation, changing $h$ changes the visible detail while preserving the total integral one. For a Gaussian density kernel, each bump has standard deviation $h$ and its individual contribution has peak height $1/(nh\sqrt{2\pi })$.

If the smoothing bandwidth is too small relative to the spacings in a statistical sample, the kernel density estimator shows narrow spikes tied to individual observations: apparent peaks may be sampling accidents. If it is too large relative to features of the unknown probability density function, it blurs distinct groups together and spreads fitted probability far from the observations. An intermediate smoothing bandwidth can reveal broad structure, but visual smoothness alone cannot establish accuracy.

The first three panels use exactly $(-1.8,-1.4,-1.1,-0.7,0.7,1.1,1.4,1.8)$ with smoothing bandwidths $0.08$, $0.35$ and $1.2$, respectively. Every curve has area one; the two groups in the observed statistical sample become less distinct as $h$ increases. No true probability density function is specified in this illustration, so the middle panel is illustrative, not a certified optimal fit. The fourth panel uses $(0.1,0.4,0.9)$ and $h=0.5$ to illustrate boundary bias of a kernel density estimator.

For a fixed nonnegative kernel for density estimation $K$ with finite squared integral, a probability density function $f$ that is bounded and continuous at an interior point $x$ with $f(x)>0$, and a deterministic sequence $h=h_n\downarrow 0$, the pointwise sampling variance obeysThus decreasing the smoothing bandwidth increases the leading sampling variance while reducing interior smoothing bias of an estimator: this is a bias-variance tradeoff. The conditions $h_n\to 0$ and $n{h}_n\to \infty$ express two goals, increasingly fine resolution and enough observations contributing locally. Practical methods such as plug-in bandwidth selection use the data to choose $h$; pointwise optimal kernel bandwidth depends on local features of the unknown probability density function. The smoothing bandwidth remains a tuning choice in nonparametric statistics, and knowing the support of a probability distribution can require more than a width adjustment near a boundary.

Estimate the unknown curvature in the AMISE minimizer and substitute it into that formula. A consistent positive curvature estimate makes the selected bandwidth ratio-consistent. Pilot smoothing is needed because differentiating an undersmoothed density estimator magnifies variance. A normal-reference pilot can initialize an estimate of a higher derivative functional; the final density-curvature estimate is computed from the data rather than assuming the density is normal.

Twice integrating by parts gives $R(f^{\prime \prime })=\int f{f}^{(4)}$. Estimate this by averaging a fourth-derivative kernel density estimator at the observations. Its expectation contains diagonal term $K^{(4)}(0)/(n{g}^5)$ and leading smoothing bias $-g^2{\mu }_2(K)R(f^{\prime \prime \prime })/2$. Balancing them yields pilot scale $g\asymp R(f^{\prime \prime \prime }{)}^{-1/7}{n}^{-1/7}$. With suitable derivatives and tails, the estimation error is bounded by $O_p(g^2+(n{g}^5{)}^{-1}+n^{-1/2}+n^{-1}{g}^{-9/2})$, giving an $O_p(n^{-2/7})$ relative-bandwidth bound after substitution. More carefully calibrated bias cancellation can improve that bound.

// Target: statistical-inference.bigb

The uniform smoothing kernel assigns equal weight inside the window $[x-h,x+h]$ and zero weight outside. Its Nadaraya–Watson estimator is the average of observations in that window.

The unit-width centred box kernel is $K(u)=1_{[-1/2,1/2]}(u)$. It has integral one and squared $L^2$ norm one. Its rescaling $K_h(u)=h^{-1}K(u/h)$ averages a function on intervals of length $h$. It is obtained from the uniform smoothing kernel with support $[-1,1]$ by rescaling, and is useful in both kernel density estimation and local polynomial regression.