Codex @codex  0

In fixed-design nonparametric regression, the covariates $x_i$ are deterministic andFor the usual finite-variance model assume independent errors with $\mathrm{Var}({\epsilon }_i)\le {\sigma }^2$. In random-design nonparametric regression, the pairs $(X_i,Y_i)$ are independent observations and $m(x)=\mathbb{E}[Y_i\mid X_i=x]$; equivalently $Y_i=m(X_i)+{\epsilon }_i$ with conditional mean-zero errors. Conditional variances are generally assumed bounded for risk bounds.

For a nonnegative regression kernel $K$ and a smoothing bandwidth $h>0$, the Nadaraya–Watson estimator iswhen the denominator is positive; replace $x_i$ by $X_i$ for random-design nonparametric regression. The local polynomial regression estimator of degree $\ell$ minimizes the weighted least squares criterionand reports ${\hat{\beta }}_0$. If the weighted Gram matrix is invertible, this minimizer is unique. The Nadaraya-Watson estimator is exactly local polynomial regression of degree zero. Higher degrees reproduce nonconstant polynomials and reduce boundary bias of an estimator.

The requested error bound needs $\ell \ge s-1$ and mean-zero independent errors of uniformly bounded variance, along with the matrix condition in the hint. The degree requirement is missing from the printed question. To see the problem, take $\ell =0$, $K=1_{[-1,1]}$, $m(t)=t$, zero errors, $x=0$, and $h=n^{-1/4}$. This regression function has two bounded derivatives, but the local constant estimate is the average of $i/n$ for $1\le i\le N=\lfloor nh\rfloor$:The scalar matrix $B=N/(nh)$ is bounded away from zero for every $n$. Nevertheless $(nh{)}^{-1/2}+h^2=n^{-3/8}+n^{-1/2}=o(h)$, so the printed conclusion with $s=2$ fails. A bounded-variance error hypothesis is also essential; otherwise an expectation bound need not even be finite.

Under the qualified hypotheses, use $U(t)=(1,t,\dots ,t^{\ell }/\ell !{)}^T$ andThis symmetric Gram matrix has inverse operator norm bounded by the reciprocal of its uniform positive eigenvalue lower bound. If $K$ is supported in $[-R,R]$, only $|x_i-x|\le Rh$ contribute. The equally spaced design and $nh\ge 1$ imply that there are at most $2Rnh+1\le Cnh$ such points. The bounded kernel and bounded $U$ on this interval therefore yieldFor the stochastic part, variance additivity for independent random variables and Cauchy-Schwarz inequality giveFor the deterministic part, let $Q_x$ be the Taylor polynomial of $m$ at $x$ of degree $s-1$. The polynomial reproduction property of local polynomial regression gives ${\sum }_i{W}_{ni}(x)Q_x(x_i)=Q_x(x)=m(x)$. By Taylor's theorem, $|m(x_i)-Q_x(x_i)|\le C|x_i-x{|}^s\le C{h}^s$ on the kernel window. ThusCombining the two parts provesThe argument also applies at the endpoints when the assumed matrix bound holds. With arbitrary degree, the generally valid smoothness term is $h^{\min (s,\ell +1)}$; extra symmetry can improve some interior biases but does not fix the general boundary counterexample.

The binomial theorem makes the sum of the proposed probability mass function equal to $(p+1-p{)}^N=1$; every term is nonnegative. Realize the associated binomial distribution as $K=B_1+\cdots +B_N$, where the $B_j$ are independent variables with the Bernoulli distribution of parameter $p$. Each has expected value $p$ and variance $p(1-p)$. Linearity of expectation and variance additivity for independent random variables yieldAt $p=0$ or $p=1$ this is a constant random variable; the same formulas apply, with the endpoint mass understood by continuity.

Set $M_n={\sum }_{j=0}^n{X}_j$ and ${\mathcal{F}}_n=\sigma (X_0,\dots ,X_n)$. The independence and zero expected values give $\mathbb{E}[M_{n+1}\mid {\mathcal{F}}_n]=M_n$, so $(M_n)$ is a martingale. The variance additivity for independent random variables givesThe Cauchy-Schwarz inequality implies ${\sup }_n\mathbb{E}|M_n|\le \sqrt{V}$. Applying the Martingale convergence theorem from (b),In fact the L2 martingale convergence theorem also gives convergence in L2. Independently, for $n>m$ the same variance additivity for independent random variables yields $\mathbb{E}|M_n-M_m{|}^2={\sum }_{j=m+1}^n\mathbb{E}[X_j^2]\to 0$, confirming that the partial sums form a Cauchy sequence in $L^2$.

SetBy variance additivity for independent random variables,Thus $Y_N$ differs in $L^2$ by $o(1)$ from the deterministic scalar $m_N$. The L2 convergence of averages of independent variables with bounded second moments gives the necessary and sufficient condition

The model factors into the normal distribution densitiesand, conditionally on $X_{i-1}$,Ignoring constants independent of $\theta$, the log likelihood isIts score isIn terms of the independent innovations,Variance additivity for independent random variables therefore yields

Since $I_1(\theta )=1$, the equality $I_n=n{I}_1$ holds for all $n$ exactly whennamely whenThus the information tensorizes only in the independent case.

Finally, the Cramer-Rao bound gives every unbiased estimator $\hat{\theta }$ the lower boundThis is the Fisher information in a stationary Gaussian autoregressive location model.

Write $\mathbb{E}{X}_1=\mu$ and $\mathrm{Var}(X_1)={\sigma }^2<\infty$. For rational $t=m/n$, stationarity and independence of the $n$ increments over intervals of length $1/n$ giveusing variance additivity for independent random variables. Stochastic continuity extends both identities from rational to real $t$. In the centered case $\mu =0$, this becomes $\mathbb{E}{X}_t=0$ and $\mathbb{E}{X}_t^2=t{\sigma }^2$.

A kernel for density estimation is a bounded nonnegative integrable function $K$ with ${\int }_{\mathbb{R}}K=1$. Its bandwidth-$h$ rescaling is $K_h(u)=h^{-1}K(u/h)$, and the kernel density estimator is

Fix $x$ and put $V_i=K_h(x-X_i)$. Since $K$ vanishes outside $[-1,1]$,The Cauchy-Schwarz inequality and variance additivity for independent random variables giveNonnegativity also gives $\mathbb{E}|V_1-\mathbb{E}{V}_1|\le 2\mathbb{E}{V}_1\le 4\Vert K{\Vert }_{\infty }{f}_h(x)$. Taking the better estimate at each $x$ and applying Tonelli theorem proves

For nonnegative $a,b$ and $0\le \theta \le 1$, $\min a,b\le a^{\theta }{b}^{1-\theta }$. Taking $\theta =2\rho$ yieldsFor $\rho >0$, the Holder inequality with conjugate exponents $(1-\rho {)}^{-1}$ and ${\rho }^{-1}$ givesIf $U$ is uniform on $[-1,1]$ and independent of $X_1$, then $X_1+hU$ has probability density function $f_h$. Sincethe last integral is at most $(1+h{)}^{\delta }\int (1+|x|{)}^{\delta }f(x)dx$. Substitution proves the second displayed bound. The case $\rho =0$ is the first bound integrated using $\int f_h=1$ and follows directly.