= Solution
Put
$$
r_p(u)=(1,u,\ldots,u^p)^T,
\qquad
u_i=\frac{x_i-x}{h},
\qquad
K_i=K(u_i).
$$
The degree-$p$ <local polynomial estimator> is $\widehat m_n(x)=\widehat b_0$, where
$$
\widehat b\in\operatorname*{argmin}_{b\in\mathbb R^{p+1}}
\sum_{i=1}^n
\{Y_i-b^Tr_p(u_i)\}^2K_i.
$$
Define the <local polynomial Gram matrix>
$$
B_{p,n}(x)=\frac1{nh}\sum_{i=1}^n
r_p(u_i)r_p(u_i)^TK_i.
$$
Assume that it is positive definite, and write
$$
\lambda_0=\lambda_{\min}\{B_{p,n}(x)\}>0.
$$
The normal equations then give
$$
\widehat m_n(x)
=\frac1n\sum_{i=1}^nw_{p,i}(x)Y_i,
\qquad
w_{p,i}(x)
=\frac1h e_1^TB_{p,n}(x)^{-1}r_p(u_i)K_i.
$$
Thus the estimator is a <linear estimator in nonparametric regression>, and the displayed $w_{p,i}$ are its <effective kernel weights>.
If $R$ is a polynomial of degree at most $p$, then $R(x+hu)=c^Tr_p(u)$ for some $c$. Feeding $Y_i=R(x_i)$ into the weighted least-squares problem gives the exact zero-residual fit $c$, whose intercept is $R(x)$. Hence the <polynomial reproduction property of local polynomial regression> is
$$
\frac1n\sum_{i=1}^nw_{p,i}(x)R(x_i)=R(x).
$$
Let $\beta_0=\lceil\beta\rceil-1$. The <Hölder class> $\mathcal H(\beta,L)$ consists of functions with derivatives through order $\beta_0$ bounded by $L$ and
$$
|f^{(\beta_0)}(s)-f^{(\beta_0)}(t)|
\leq L|s-t|^{\beta-\beta_0}.
$$
The subclass $\widetilde{\mathcal H}(\beta,L)$ additionally makes every derivative of order below $\beta_0$ $L$-Lipschitz.
Suppose $p<\beta_0$. Taylor's theorem and that additional Lipschitz condition give, for the degree-$p$ Taylor polynomial $T_p$ at $x$,
$$
|m(x_i)-T_p(x_i)|
\leq\frac{L}{(p+1)!}|x_i-x|^{p+1}.
$$
On the support of $K_i$, $|u_i|\leq1$, so
$$
|w_{p,i}(x)|
\leq\frac{\sqrt{p+1}\,\|K\|_\infty}{h\lambda_0}.
$$
For the regular design $x_i=i/n$, at most $2nh+1\leq4nh$ points satisfy $|x_i-x|\leq h$ when $h\geq1/(2n)$. Polynomial reproduction cancels $T_p$, and therefore
$$
\begin{aligned}
|\operatorname{Bias}\widehat m_n(x)|
&\leq\frac1n\sum_i|w_{p,i}(x)|
|m(x_i)-T_p(x_i)|\\
&\leq
\frac{4L\sqrt{p+1}\,\|K\|_\infty}
{(p+1)!\lambda_0}h^{p+1}.
\end{aligned}
$$
Thus the universal exponent in the question is $\gamma=1$, with the displayed choice of $C(\lambda_0,p,L,K)$.
Back to article page
