Nadaraya–Watson estimator

This local-constant estimator is one particular kernel regression method.

The Nadaraya–Watson estimator is the degree-zero local polynomial estimateAt a support boundary it generally has first-order bias because it reproduces constants but not linear functions.

For the unit-width symmetric box kernel for density estimation, suppose the design density is positive and differentiable at $x$, the regression function is twice differentiable there, and the conditional variance and design density are bounded near $x$. Write $G_n=(nh{)}^{-1}{\sum }_i(Y_i-m(x))1_{\{|X_i-x|\le h/2\}}$. Symmetry cancels its linear Taylor term, giving $\mathbb{E}{G}_n=O(h^2)$; independence gives $\mathrm{Var}{G}_n=O((nh{)}^{-1})$. The absolute error bound for a ratio estimator with a controlled denominator gives the displayed rate if the small-denominator contribution is negligible. The balance $h\asymp n^{-1/5}$ gives $O(n^{-2/5})$. Interior design positivity matters: support boundaries can prevent the linear cancellation.

A local constant average reproduces constants but need not reproduce a linear regression function near an endpoint. For $x_i=i/n$, $K=1_{[-1,1]}$, $m(x)=x$, and $nh\to \infty$, its deterministic estimate at zero is $(\lfloor nh\rfloor +1)/(2n)\sim h/2$. A local linear fit removes this first-order boundary bias of an estimator.

For fixed $\alpha \in [0,1)$, design $x_i=i/n$, $m\in C^2[0,1]$, a nonzero nonnegative symmetric piecewise $C^1$ regression kernel supported on $[-1,1]$, and $h\to 0$, $n{h}^2\to \infty$, write ${\mu }_{r,\alpha }={\int }_{-\alpha }^1{u}^{r}K(u)du$ and ${\beta }_r={\mu }_{r,\alpha }/{\mu }_{0,\alpha }$. On the rescaled grid of spacing $1/(nh)$, the Riemann sum error for $u^{r}K(u)$ is $O((nh{)}^{-1})$, by bounded variation. The normalized first and second local moments are therefore $h{\beta }_1+O(n^{-1})$ and $h^2{\beta }_2+O(h/n)$. A uniform second-order Taylor expansion gives the displayed bias of an estimator, since $n^{-1}=o(h^2)$. Equivalently it is $h{\beta }_1{m}^{\prime }(0)+h^2(\alpha {\beta }_1+{\beta }_2/2)m^{\prime \prime }(0)+o(h^2)$. The first-order term explains the generic boundary bias of a local constant estimator.

For an equally spaced design with independent errors of variance at most ${\sigma }^2$ and a regression function with Lipschitz constant $L$, the unit-width box kernel estimator averages the $N_x$ observations in the window. Its bias is at most $Lh/2$, and its variance is at most ${\sigma }^2/N_x$. The window occupancy for an equally spaced regression design gives, whenever $nh\ge 2$,In the usual range $h\le 1$, this is $O(\sigma /\sqrt{nh}+Lh)$ uniformly up to the boundary. For arbitrarily large bandwidth, the stochastic error cannot continue to decrease as $1/\sqrt{nh}$: once all observations are included the estimator is simply their mean.

At an interior point where the regression function is differentiable, a symmetric compactly supported regression kernel makes the first-order bias of an estimator vanish asymptotically. In regular fixed-design nonparametric regression with $h\to 0$ and $nh\to \infty$, the local constant estimator then has squared bias of an estimator $o(h^2)$ and variance $R(K)/(nh)+o((nh{)}^{-1})$ for a unit-integral regression kernel. The differentiability remainder is pointwise and need not hold uniformly over a function class.