Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 210 / 3

Write $t_i=(x_i-x)/h$, $r_p(t)=(1,t,\dots ,t^p{)}^T$, and defineThe degree-$p$ local polynomial regression fit minimizesAssume its local polynomial Gram matrixis positive definite. The normal equations then give

Because the polynomial is written in the scaled coordinate, the local polynomial derivative estimator is ${\hat{m}}_n^{\prime }(x)={\hat{\beta }}_1/h$. If $e_1=(0,1,0,\dots ,0{)}^T$, thenFor a polynomial $R$ of degree at most $p$, the local least-squares fit to $Y_i=R(x_i)$ is exactly the Taylor polynomial $R(x+ht)$, so its scaled linear coefficient is $h{R}^{\prime }(x)$. Therefore the polynomial reproduction property of local polynomial regression gives

Positive definiteness and the fixed finite-dimensional basis provide a number ${\lambda }_0>0$ such thatSince $K(t_i)=0$ outside $|x_i-x|\le h$,and

The regular design has at most $2nh+2\le 4nh$ points in this window when $h\ge 1/(2n)$. Since the errors are independent with variance at most ${\sigma }^2$,

Finally let ${\beta }_0=\lceil \beta \rceil -1$ and take the degree-${\beta }_0$ Taylor polynomial $R$ of $m$ at $x$. The Hölder class remainder satisfiesPolynomial reproduction removes $R$ from the bias. On the kernel window the remainder is at most $L{h}^{\beta }/{\beta }_0!$, so the weight-sum bound gives