Bias of a kernel density estimator

For a symmetric bandwidth-$h$ averaging kernel and a density with bounded first derivative, the interior pointwise bias is at most a constant times $h\Vert f^{\prime }{\Vert }_{\infty }$. Near a support boundary, an uncorrected symmetric kernel can instead have nonvanishing bias.

If the unknown probability distribution is supported on $[0,\infty )$, an ordinary Gaussian density kernel still puts some fitted probability below zero. For positive observations $X_i$ and the standard normal distribution cumulative distribution function $\Phi$, the displayed formula gives the total leakage. For the statistical sample $(0.1,0.4,0.9)$ and smoothing bandwidth $h=0.5$, it isThus about $22.28\%$ of the fitted probability falls outside the known support of a probability distribution, even though the full integral remains one. The leakage panel in smoothing bandwidth shades this area. Simply deleting the negative part leaves area $0.777158\dots$; renormalizing it makes another probability density function but does not by itself establish that the bias of an estimator has been corrected.

The issue also appears in the expectation of the kernel density estimator. For a nonnegative symmetric kernel for density estimation and a probability density function $f$ that is zero for $x<0$ and has a finite positive right-hand limit $f(0+)$, assume $f$ is bounded. Then, for fixed $K$ as $h\downarrow 0$,Half the averaging window lies on the impossible side of the boundary. For example, the rate-one exponential distribution has $f(0+)=1$, so the limiting expected boundary height is $1/2$, rather than the right-hand height one. This refers to estimating the right-hand limit; changing a probability density function at a single point does not change its probability distribution. Decreasing the smoothing bandwidth narrows the affected region but does not remove this boundary-height error. This is why the interior second-order pointwise bias bound for kernel density estimation cannot be applied across a jump at the boundary.

For a nonnegative symmetric unit-integral kernel for density estimation and $f\in H^2(\mathbb{R})$, the integral Taylor remainder and Minkowski inequality give $\Vert K_h\ast f-f{\Vert }_2\le h^2{\mu }_2(K)\Vert f^{\prime \prime }{\Vert }_2/2$. Translation preserves the L2 norm. For a signed kernel the same proof requires the absolute moment $\int u^2|K(u)|du$.

An uncorrected nonnegative symmetric kernel density estimator can leak an order-one amount across a jump in a probability density function. If its transition region has width proportional to the smoothing bandwidth $h$, the integrated mean squared error includes squared bias of an estimator of order $h$, rather than the order $h^4$ familiar for twice-smooth interior densities. For the unit-width uniform window and the exponential distribution of rate one, the integrated squared bias of an estimator is $h/12+o(h)$.

For a density with finitely many jumps ${\Delta }_r$ at separated points and the unit-width box kernel, take $h$ smaller than all consecutive jump spacings. The smoothing bias is supported in disjoint length-$h$ neighbourhoods of the jumps. At each jump its two sides form triangles of height $|{\Delta }_r|/2$, and exact integration givesCombined with the integrated variance of a kernel density estimator, this gives expected $L^2$ error $O(n^{-1/4})$ at $h\asymp n^{-1/2}$.

Let a nonnegative kernel for density estimation $K$ have integral one, zero first moment and finite second moment ${\mu }_2(K)$. If a probability density function $f$ has bounded second derivative, its kernel density estimator has pointwise bias at most $h^2{\mu }_2(K)\Vert f^{\prime \prime }{\Vert }_{\infty }/2$. Apply the Taylor theorem with Lagrange remainder to $f(x-hu)$ and integrate: the first-order term vanishes and the remainder is bounded by $h^2{u}^2\Vert f^{\prime \prime }{\Vert }_{\infty }/2$. For $K=1_{[-1/2,1/2]}$, ${\mu }_2(K)=1/12$.