= Solution
For a <kernel for density estimation> $K$ with integral one and a <smoothing bandwidth> $h>0$, the univariate <kernel density estimator> is
$$
\boxed{K_h(u)=\frac1hK(u/h),\qquad
\widehat f_h(x)=\frac1n\sum_{i=1}^nK_h(x-X_i).}
$$
Its <expected value> is
$$
\mathbb E\widehat f_h(x)
=\frac1n\sum_{i=1}^n\int_{\mathbb R}K_h(x-y)f(y)\,dy
=\int_{\mathbb R}K_h(x-y)f(y)\,dy
=(K_h*f)(x).
$$
This is the <convolution> identity. It holds wherever the integral is defined, in particular everywhere for the bounded kernel used below. For a nonnegative <kernel for density estimation>, <Tonelli theorem> also gives the identity as an extended nonnegative integral. A signed <kernel for density estimation> instead requires absolute integrability for this interchange.
For the centred unit-width <uniform kernel>, substitute $u=(x-y)/h$:
$$
\mathbb E\widehat f_h(x)=\int_{-1/2}^{1/2}f(x-hu)\,du.
$$
Put $M=\sup_z|f''(z)|$. The <Taylor theorem with Lagrange remainder> gives
$$
f(x-hu)=f(x)-hu f'(x)+R(x,u),\qquad
|R(x,u)|\leq\frac{Mh^2u^2}{2}.
$$
The <odd function> term integrates to zero, while $\int_{-1/2}^{1/2}u^2\,du=1/12$. Hence the <bias of a kernel density estimator> obeys
$$
\boxed{\left|\mathbb E\widehat f_h(x)-f(x)\right|\leq\frac{Mh^2}{24}.}
$$
This is a <second-order pointwise bias bound for kernel density estimation>. Existence and boundedness of the <second derivative> suffice for the stated remainder; <continuity> of that <derivative> is not an extra assumption.
For concentration, write
$$
B_i=\mathbb1_{\{|X_i-x|\leq h/2\}},\qquad
p=\mathbb P(|X_1-x|\leq h/2).
$$
The $B_i$ are <independent and identically distributed random variables> with a <Bernoulli distribution>, and $\widehat f_h(x)=(nh)^{-1}\sum_iB_i$. The required concentration bound follows from the <Hoeffding lemma>. We prove its Bernoulli case directly. With
$$
\psi(s)=\log\mathbb E e^{s(B_1-p)},
$$
we have $\psi(0)=\psi'(0)=0$ and $\psi''(s)=p_s(1-p_s)\leq1/4$, where $p_s=pe^s/(1-p+pe^s)$ is the exponentially tilted success <probability>. Therefore $\psi(s)\leq s^2/8$ for every real $s$. If $p=0$ or $p=1$, the centred variable is identically zero and the same bound holds directly.
<Independence> and the exponential form of the <Markov inequality> now give, for $s>0$,
$$
\mathbb P\!\left(\sum_i(B_i-p)\geq nht\right)
\leq\exp\!\left(-snht+\frac{ns^2}{8}\right).
$$
Choose $s=4ht$. Applying the same argument to the lower tail and adding the probabilities proves the <Hoeffding inequality>
$$
\boxed{\mathbb P\!\left(\left|\widehat f_h(x)-\mathbb E\widehat f_h(x)\right|\geq t\right)
\leq2e^{-2nh^2t^2}.}
$$
To obtain the requested <variance> constant, use the additional bound that a <probability> is at most one. Set $Z=\widehat f_h(x)-\mathbb E\widehat f_h(x)$ and $a=2nh^2$. The <tail integral formula for moments>, justified by <Tonelli theorem>, yields
$$
\begin{aligned}
\operatorname{Var}(\widehat f_h(x))
&=\mathbb E Z^2
=\int_0^\infty\mathbb P(Z^2>u)\,du\\
&\leq\int_0^\infty\min\{1,2e^{-au}\}\,du\\
&=\frac{\log2}{a}+\int_{\log2/a}^{\infty}2e^{-au}\,du
=\frac{1+\log2}{a}.
\end{aligned}
$$
Thus
$$
\boxed{\operatorname{Var}(\widehat f_h(x))\leq\frac{1+\log2}{2nh^2}.}
$$
The clipping at one is essential for this integration argument to give that constant. It is an instance of a <second moment bound from a clipped Gaussian tail>. The exact <Bernoulli distribution> calculation also gives $\operatorname{Var}(\widehat f_h(x))=p(1-p)/(nh^2)\leq1/(4nh^2)$, an even stronger bound.
Back to article page
