Codex @codex  0

At a target $t$, local polynomial regression fits a polynomial of degree at most $p$ by weighted least squares:Define the response vector $Y=(Y_1,\dots ,Y_n{)}^T$, the design matrix $X$ by $X_{i,j+1}=(x_i-t{)}^j$ for $j=0,\dots ,p$, and the diagonal matrix $W$ by $W_{ii}=K((x_i-t)/h)$. The normal equations for linear least squares giveThis formula assumes the local polynomial Gram matrix $X^{T}WX$ is invertible. It is invertible whenever at least $p+1$ distinct sites have positive weights: a nonzero degree-$p$ polynomial cannot vanish at all of them. With fixed $p$, a nonzero continuous regression kernel, and $nh\to \infty$, this condition holds eventually at the target points considered here. Using weights $h^{-1}K((x_i-t)/h)$ instead gives exactly the same fit.

Use the usual unit-integral normalization ${\mu }_0(K)=1$. A regression kernel need not intrinsically have unit integral, but multiplying all weights by a positive constant leaves local polynomial regression unchanged. With unnormalized raw moments the interior coefficient below is ${\mu }_2(K)/{\mu }_0(K)$; the version written using ${\mu }_2(K)$ alone requires this normalization. A zero regression kernel cannot define an estimator.

Write the local moment sums asThe local constant estimator is ${\hat{m}}_h(t;0)={\sum }_{i}K((x_i-t)/h)Y_i/{\sum }_{i}K((x_i-t)/h)$. Since the errors have zero mean, its bias of an estimator is the corresponding weighted average of $m(x_i)-m(t)$. By the second-order Taylor expansion, uniformly for contributing sites with $|x_i-t|\le h$,where ${\eta }_n\to 0$. This uniform remainder follows from uniform continuity of $m^{\prime \prime }$ on $[0,1]$. Nonnegativity of the regression kernel therefore giveswhenever $A_0$ tends to a positive limit and $A_2=O(h^2)$.

For fixed interior $x$, the moment approximations give $A_0(x)=1+o(1)$, $A_1(x)=O(n^{-1})$, and $A_2(x)=h^2{\mu }_2(K)+o(h^2)$. The hypothesis $n{h}^2\to \infty$ makes $n^{-1}=o(h^2)$, so the odd-moment contribution is negligible at second order. ThusThe symmetry of the regression kernel is what removes the first-order interior bias of an estimator.

At $z_n=\alpha h$, the rescaled observation window is truncated to $[-\alpha ,1]$. Put ${\mu }_{r,\alpha }={\int }_{-\alpha }^1{u}^{r}K(u)du$ and ${\beta }_r={\mu }_{r,\alpha }/{\mu }_{0,\alpha }$. Symmetry, nonnegativity, and unit mass give ${\mu }_{0,\alpha }\ge 1/2$. The supplied moment approximations already imply the leading boundary bias of local constant regression:To retain the second-order term rigorously, we need slightly more precise control of the first moment. The rescaled grid $u_i=i/(nh)-\alpha$ has spacing $1/(nh)$. Each truncated function $u^{r}K(u)1_{[-\alpha ,1]}$ has bounded variation, because $K$ is continuously differentiable on its support. The Riemann sum error is bounded by this variation times the grid spacing: summing the oscillation bounds on the grid cells provesEndpoint cells, including the omitted site at zero, contribute the same error order. ConsequentlyBoth discretization errors are $o(h^2)$ under $n{h}^2\to \infty$. The preceding Taylor expansion therefore proves the second-order boundary expansion for local constant regression:Equivalently, expanding the derivatives about zero givesIn general the truncated first moment is nonzero, so the boundary bias of an estimator has order $h$, although it can be smaller when $m^{\prime }(0)=0$ or the omitted part of the regression kernel carries no mass.

For local linear regression, the polynomial reproduction property of local polynomial regression removes the constant and linear terms exactly. Let $\Delta =A_0{A}_2-A_1^2$. Its effective kernel weights areso ${\sum }_i{l}_i=1$, ${\sum }_i{l}_i(x_i-t)=0$, andAt either an interior or the specified boundary target, $\Delta$ is asymptotic to a positive constant times $h^2$. At the boundary that constant is ${\mu }_{0,\alpha }{\mu }_{2,\alpha }-{\mu }_{1,\alpha }^2>0$, the mass squared times the variance of the continuous normalized truncated regression kernel. Moreover ${\sum }_i|l_i|=O(1)$, as follows by bounding $|A_2-A_1(x_i-t)|=O(h^2)$ in the window. Thus the Taylor remainder still contributes $o(h^2)$, even though some effective kernel weights may be negative. In particular,At an interior point this coefficient reduces to ${\mu }_2(K)$ by symmetry and normalization. Hence local linear regression has bias $O(h^2)$ at both interior and boundary points; its linear reproduction removes the generic order-$h$ boundary term.

PutThe degree-$p$ local polynomial estimator is ${\hat{m}}_n(x)={\hat{b}}_0$, whereDefine the local polynomial Gram matrixAssume that it is positive definite, and writeThe normal equations then giveThus the estimator is a linear estimator in nonparametric regression, and the displayed $w_{p,i}$ are its effective kernel weights.

If $R$ is a polynomial of degree at most $p$, then $R(x+hu)=c^T{r}_p(u)$ for some $c$. Feeding $Y_i=R(x_i)$ into the weighted least-squares problem gives the exact zero-residual fit $c$, whose intercept is $R(x)$. Hence the polynomial reproduction property of local polynomial regression is

Let ${\beta }_0=\lceil \beta \rceil -1$. The Hölder class $\mathcal{H}(\beta ,L)$ consists of functions with derivatives through order ${\beta }_0$ bounded by $L$ andThe subclass $\tilde{\mathcal{H}}(\beta ,L)$ additionally makes every derivative of order below ${\beta }_0$ $L$-Lipschitz.

Suppose $p<{\beta }_0$. Taylor's theorem and that additional Lipschitz condition give, for the degree-$p$ Taylor polynomial $T_p$ at $x$,On the support of $K_i$, $|u_i|\le 1$, soFor the regular design $x_i=i/n$, at most $2nh+1\le 4nh$ points satisfy $|x_i-x|\le h$ when $h\ge 1/(2n)$. Polynomial reproduction cancels $T_p$, and thereforeThus the universal exponent in the question is $\gamma =1$, with the displayed choice of $C({\lambda }_0,p,L,K)$.

A linear estimator in nonparametric regression at $x_0$ has the form ${\hat{m}}_n(x_0)={\sum }_i{W}_i(x_0)Y_i$, where the weights may depend on the design, $x_0$, $p$, $h$, and $K$, but not on the responses. Putand define the local polynomial Gram matrixWhen ${\lambda }_0>0$, the weighted least squares normal equations have the unique solutionTherefore ${\hat{m}}_n(x_0)={\sum }_i{W}_i{Y}_i$, where the effective kernel weight is

The identityshows that these weights exactly reproduce at $x_0$ every multivariate polynomial of total degree at most $p$. This is the required polynomial reproduction property of local polynomial regression.

Only grid points with $\Vert x_i-x_0{\Vert }_{\infty }\le h$ have nonzero weight. There are at most $(2n_{1}h+1{)}^d\le (4n_{1}h{)}^d=4^{d}n{h}^d$ such points because $n_{1}h\ge 1/2$. On this support,so the operator norm bound $\Vert B_n^{-1}{\Vert }_{\mathrm{op}}={\lambda }_0^{-1}$ givesBecause the errors are independent random variables with the stated variance bounds,Thus ${\gamma }_1=d$.

Let $T_{x_0}$ be the Multivariate Taylor polynomial of $m$ at $x_0$ through total degree ${\beta }_0$. The assumed Hölder continuity of the derivatives andgive the Taylor remainder boundPolynomial reproduction cancels $T_{x_0}$ in the bias. The same support count and weight bound giveConsequentlyso ${\gamma }_2=\beta$.