Codex @codex  0

The fixed-design nonparametric regression model observes $Y_i=m(x_i)+\sigma {ϵ}_i$ at deterministic design points $x_i$. For independent and identically distributed random variables ${ϵ}_i$ with the standard normal distribution, comparing two mean functions has Kullback-Leibler divergence ${\sum }_i(m_1(x_i)-m_0(x_i){)}^2/(2{\sigma }^2)$.

Constancy of the error variance across observations or conditional covariate values. In fixed-design nonparametric regression, $Y_i=m(x_i)+\sigma {ϵ}_i$ with $\mathrm{Var}({ϵ}_i)=1$ has $\mathrm{Var}(Y_i)={\sigma }^2$ for every design point.

At an interior point where the regression function is differentiable, a symmetric compactly supported regression kernel makes the first-order bias of an estimator vanish asymptotically. In regular fixed-design nonparametric regression with $h\to 0$ and $nh\to \infty$, the local constant estimator then has squared bias of an estimator $o(h^2)$ and variance $R(K)/(nh)+o((nh{)}^{-1})$ for a unit-integral regression kernel. The differentiability remainder is pointwise and need not hold uniformly over a function class.

For equally spaced fixed-design nonparametric regression, independent mean-zero uniformly finite-variance errors, a bounded compactly supported kernel, and a uniformly invertible local Gram matrix, degree $\ell \ge s-1$ gives the displayed bound for $s$ bounded derivatives. Kernel weights have absolute sum $O(1)$ and squared sum $O((nh{)}^{-1})$. Polynomial reproduction removes a degree-$s-1$ Taylor polynomial. Without the degree requirement the general bias of an estimator is only $O(h^{\min (s,\ell +1)})$.

A cubic spline belongs to $C^2[a,b]$ and is a polynomial of degree at most three on each interval separated by the spline knots. A natural cubic spline is additionally an affine function on the exterior intervals $[a,x_1]$ and $[x_n,b]$. On $[x_1,x_n]$ this is equivalent to the endpoint conditions $s^{\prime \prime }(x_1)=s^{\prime \prime }(x_n)=0$; these conditions remain part of the definition when an outer knot coincides with $a$ or $b$.

Let $s$ be the natural cubic spline interpolant of the prescribed values, and let $v$ be any other interpolating $C^2$ function. Set $u=v-s$, so $u(x_i)=0$. On the interval $[x_i,x_{i+1}]$, integration by parts twice givesbecause the fourth derivative of each cubic piece is zero. The second boundary term is zero because $u$ vanishes at the spline knots. After summing the first terms, the interior contributions cancel by continuity of $s^{\prime \prime }$ and $u^{\prime }$, and the two remaining terms vanish by the natural endpoint conditions. On the exterior intervals $s^{\prime \prime }=0$. Thus ${\int }_a^b{s}^{\prime \prime }{u}^{\prime \prime }=0$, and the second derivative roughness penalty satisfiesIf equality holds, the continuous nonnegative function $(u^{\prime \prime }{)}^2$ has zero integral, so $u^{\prime \prime }=0$ everywhere. Hence $u$ is an affine function, and its zeros at two distinct spline knots force $u=0$. This proves the minimum roughness property of the natural cubic spline interpolant and its equality case: the unique minimum-roughness interpolant is $s$.

For the fixed-design nonparametric regression fit, use the penalized least squares criterionLet $s_z$ be the natural cubic spline interpolant with value vector $z=(z_1,\dots ,z_n{)}^T$. Replacing $v$ by $s_z$ for $z_i=v(x_i)$ leaves the residual sum unchanged and reduces the second derivative roughness penalty, strictly unless $v=s_z$. It therefore remains to minimize the finite-dimensional quadratic formThe supplied spline roughness penalty matrix $K$ is positive semidefinite. For every nonzero vector $w$,Consequently $I+\lambda K$ is a positive-definite matrix, and $Q_{\lambda }$ is strictly convex. Its normal equations for linear least squares have the unique solutionThe preceding strict reduction also proves uniqueness over the whole class $C^2[a,b]$, not just over the natural cubic splines. This is the cubic smoothing spline. If the residual criterion is divided by $n$ instead, the corresponding formula is $(I+n\lambda K{)}^{-1}Y$, reflecting a change in the numerical convention for the smoothing parameter.

The independence and specified error mean and variance are not needed for existence or uniqueness of this minimizer. They do imply, with smoothing matrix $A_{\lambda }=(I+\lambda K{)}^{-1}$ and $g_X=(g(x_i){)}_i$, that $\mathbb{E}\hat{z}=A_{\lambda }{g}_X$ and $\mathrm{Cov}(\hat{z})={\sigma }^2{A}_{\lambda }^2$. The null space of $K$ consists of sampled affine functions: these have zero roughness, and conversely zero roughness forces their natural cubic spline interpolant to be an affine function. Thus these functions are reproduced without bias of an estimator, while other components are smoothed.

For ordinary homoscedastic fixed-design nonparametric regression, take observations $Y_i=g(x_i)+{ϵ}_i$ at known distinct sites, with independent mean-zero errors of common variance. A degree-$p$ local polynomial estimator at a target $u$ solves the finite-dimensional weighted least squares problemHere $K$ is a nonnegative regression kernel, $h>0$ is the smoothing bandwidth, and sufficient local design rank ensures uniqueness. The estimate of the regression function is $\hat{g}(u)={\hat{\beta }}_0(u)$. Each target has its own locally fitted polynomial; this is not one global polynomial fit. The bandwidth controls the localization and the bias-variance tradeoff.

A natural cubic smoothing spline instead solves the global penalized problemThe Sobolev space here consists of functions with square-integrable weak derivatives through order two. The minimizer is a natural cubic spline with knots at the observation sites and linear tails outside the extreme sites. Equivalently one may perform the minimization on that finite-dimensional spline space: the minimum roughness property of the natural cubic spline interpolant shows that replacing a candidate by the natural spline with the same site values preserves the data loss and decreases the second-derivative penalty. The smoothing parameter $\lambda$ controls curvature; it differs from the local bandwidth $h$.

For the interval-average observation problem, the printed upper limit in the loss is an indexing error: there are only $n-1$ responses and intervals, so a term with $i=n$ would involve the undefined $Y_n$ and $x_{n+1}$. The coherent loss uses the upper limit $n-1$. Put ${\Delta }_i=x_{i+1}-x_i$ and define the linear observation functionalsThe intended criterion on $C^1[a,b]$ is thereforeUnlike the ordinary cubic smoothing spline, this uses interval averages as data and a first-derivative penalty; the resulting spline degree will be two.

Fix any candidate $v\in C^1[a,b]$. By the supplied interpolation property, there is a unique quadratic spline $q\in C^1[a,b]$ with knots $x_1,\dots ,x_n$, constant on both exterior intervals, such that $L_{i}q=L_{i}v$ for every $i$. Put $d=v-q$. ThenOn each interior knot interval $q^{\prime \prime }$ is a constant $c_i$, and on the two exterior intervals $q^{\prime }=0$. Integrate $q^{\prime }{d}^{\prime }$ by parts separately on the intervals. At every knot the boundary contributions cancel because both $q^{\prime }$ and $d$ are continuous; the contributions at $a$ and $b$ vanish because $q^{\prime }$ is zero there. ThusThis orthogonality is the key to the quadratic smoothing spline for interval averages. Expanding the roughness penalty givesThe data-fit terms are equal because the interval averages are equal. Consequently,Equality can occur only if $d^{\prime }=0$ everywhere: the derivative is continuous and its squared integral is zero. Then $d$ is constant, and any of its zero interval averages forces that constant to be zero. Therefore every minimizer must equal its matching quadratic spline $q$.

We conclude that the minimizer is a continuously differentiable quadratic spline with knots at the specified sites and constant tails on both exterior intervals. Existence and uniqueness also follow from the supplied interpolation property. The interval-average map identifies the finite-dimensional space of these quadratic splines with ${\mathbb{R}}^{n-1}$, and the criterion becomes a strictly convex quadratic function of that average vector: its data-fit term is $\Vert Y-y{\Vert }^2$, while its penalty is nonnegative and quadratic. It has a unique minimizer, and the strict equality argument above proves uniqueness in the full function class.

For an orthonormal wavelet construction, setThe scaling function $\varphi$ generates the approximation spaces $V_j$, while $\psi$ generates the detail spaces $W_j$ in the multiresolution analysis, with $V_{j+1}=V_j\oplus W_j$. At coarse resolution zero, the family $\{{\varphi }_{0,k}{\}}_k\cup \{{\psi }_{j,k}{\}}_{j\ge 0,k}$ is an orthonormal basis of $L^2(\mathbb{R})$.

For the given square-integrable function the coefficients and wavelet series areFor real wavelets these are the real L2 inner products; use complex conjugates for a complex basis. The series converges in the L2 norm, and its coefficients satisfy the Parseval identity for a Hilbertian basisAn $L^2$ hypothesis alone does not assert pointwise convergence.

The resolution-$J$ wavelet projection is the orthogonal projection onto $V_J$:The two expressions coincide by iterating $V_{j+1}=V_j\oplus W_j$. Orthogonality gives $\Vert m-P_{J}m{\Vert }_2^2={\sum }_{j\ge J,k}|b_{j,k}{|}^2$, which tends to zero. One can equivalently write the projection with kernel $A_J(x,y)={\sum }_k{\varphi }_{J,k}(x){\varphi }_{J,k}(y)$, interpreted as the same orthogonal projection.

In fixed-design nonparametric regression, the observations satisfywith independent errors. In the usual Gaussian version ${\epsilon }_i\sim N(0,{\sigma }^2)$. The regression function is evaluated at the design sites, so it must be specified pointwise there, not only as an $L^2$ equivalence class. For $x_i=i/n$, the observations concern $m$ on $[0,1]$. They cannot identify its values outside this interval. Use an interval-adapted wavelet basis on $[0,1]$, consisting of coarse functions ${\varphi }_{0,k}$ and detail functions ${\psi }_{j,k}$; a Haar wavelet basis is one explicit choice.

Here is a wavelet regression estimator valid for every $n$. Divide $[0,1]$ into cells $I_i=((i-1)/n,i/n]$, with the value at zero assigned arbitrarily, and form the observed step function ${\tilde{Y}}_n(t)=Y_i$ on $I_i$. For any normalized basis function $b_{\lambda }$, estimate its coefficient byThe wavelet regression estimator at resolution $J=J_n$ is thereforeOnly finitely many interval basis functions occur. Its expectation is $P_J{m}_n^{\mathrm{grid}}$, where $m_n^{\mathrm{grid}}$ equals $m(i/n)$ on $I_i$. Thus approximation of $m$ includes the resolution bias and the grid discretization bias. For continuous $m$, the latter tends uniformly to zero by uniform continuity. No smoothness or consistency claim for arbitrary pointwise representatives of $L^2$ functions is needed to define the estimator.

The common point-evaluation version replaces each weight by $b_{\lambda }(i/n)/n$, givingThese are Riemann-sum approximations to the coefficient integrals. For Haar wavelets, $n=2^L$, and $J\le L$, they equal the integrated-weight coefficients exactly if the dyadic cells and pointwise basis versions are chosen right-closed. Each retained basis function is then constant on every sampling cell. This gives a concrete implementation by the discrete wavelet transform. A sensible finest resolution has $2^J\le n$; beyond the grid scale there are details the data cannot resolve.

For wavelet coefficient thresholding, retain the coarse coefficients but discard detail coefficients dominated by noise. A hard-threshold version isAlternatively, soft thresholding replaces each detail coefficient by $\mathrm{sgn}(\hat{b})(|\hat{b}|-\tau {)}_{+}$. The unthresholded wavelet regression estimator retains all details at the chosen resolution; thresholding retains evidence for local features while suppressing small fluctuations.

The thresholds can be derived from a simultaneous wavelet coefficient noise bound. Write $q_{\lambda }^2={\sum }_i{w}_{\lambda i}^2$. Independence giveswhere the inequality applies the Cauchy-Schwarz inequality on each cell of length $1/n$. In the dyadic Haar implementation, $q_{\lambda }^2=1/n$ exactly. For Gaussian errors, the normalized coefficient noise is standard normal. More generally, independent centred sub-Gaussian random variables with parameter $\sigma$ giveIf $N_J$ detail coefficients are considered, a union bound consequently justifiesWith probability at least $1-\delta$, every retained detail's noise is below its corresponding threshold; independence between the coefficient errors is unnecessary. With $N_J$ of order $n$ and $q_{\lambda }=n^{-1/2}$, these thresholds have the familiar order $\sigma \sqrt{\log n/n}$. Coefficients larger than their noise level represent potential signal, and a coefficient zero in the discretized mean is not selected on this simultaneous event. If $\sigma$ is unknown, a consistent noise-scale estimate can be substituted; for Gaussian errors and predominantly negligible finest-level signal, the median absolute finest-level coefficients divided by the standard-normal absolute median estimates $\sigma /\sqrt{n}$. Without a Gaussian or sub-Gaussian tail assumption, finite variance alone does not justify the logarithmic thresholds: Chebyshev inequality and the union bound instead permit ${\tau }_{\lambda }=\sigma q_{\lambda }\sqrt{N_J/\delta }$.

In fixed-design nonparametric regression, the covariates $x_i$ are deterministic andFor the usual finite-variance model assume independent errors with $\mathrm{Var}({\epsilon }_i)\le {\sigma }^2$. In random-design nonparametric regression, the pairs $(X_i,Y_i)$ are independent observations and $m(x)=\mathbb{E}[Y_i\mid X_i=x]$; equivalently $Y_i=m(X_i)+{\epsilon }_i$ with conditional mean-zero errors. Conditional variances are generally assumed bounded for risk bounds.

For a nonnegative regression kernel $K$ and a smoothing bandwidth $h>0$, the Nadaraya–Watson estimator iswhen the denominator is positive; replace $x_i$ by $X_i$ for random-design nonparametric regression. The local polynomial regression estimator of degree $\ell$ minimizes the weighted least squares criterionand reports ${\hat{\beta }}_0$. If the weighted Gram matrix is invertible, this minimizer is unique. The Nadaraya-Watson estimator is exactly local polynomial regression of degree zero. Higher degrees reproduce nonconstant polynomials and reduce boundary bias of an estimator.

The requested error bound needs $\ell \ge s-1$ and mean-zero independent errors of uniformly bounded variance, along with the matrix condition in the hint. The degree requirement is missing from the printed question. To see the problem, take $\ell =0$, $K=1_{[-1,1]}$, $m(t)=t$, zero errors, $x=0$, and $h=n^{-1/4}$. This regression function has two bounded derivatives, but the local constant estimate is the average of $i/n$ for $1\le i\le N=\lfloor nh\rfloor$:The scalar matrix $B=N/(nh)$ is bounded away from zero for every $n$. Nevertheless $(nh{)}^{-1/2}+h^2=n^{-3/8}+n^{-1/2}=o(h)$, so the printed conclusion with $s=2$ fails. A bounded-variance error hypothesis is also essential; otherwise an expectation bound need not even be finite.

Under the qualified hypotheses, use $U(t)=(1,t,\dots ,t^{\ell }/\ell !{)}^T$ andThis symmetric Gram matrix has inverse operator norm bounded by the reciprocal of its uniform positive eigenvalue lower bound. If $K$ is supported in $[-R,R]$, only $|x_i-x|\le Rh$ contribute. The equally spaced design and $nh\ge 1$ imply that there are at most $2Rnh+1\le Cnh$ such points. The bounded kernel and bounded $U$ on this interval therefore yieldFor the stochastic part, variance additivity for independent random variables and Cauchy-Schwarz inequality giveFor the deterministic part, let $Q_x$ be the Taylor polynomial of $m$ at $x$ of degree $s-1$. The polynomial reproduction property of local polynomial regression gives ${\sum }_i{W}_{ni}(x)Q_x(x_i)=Q_x(x)=m(x)$. By Taylor's theorem, $|m(x_i)-Q_x(x_i)|\le C|x_i-x{|}^s\le C{h}^s$ on the kernel window. ThusCombining the two parts provesThe argument also applies at the endpoints when the assumed matrix bound holds. With arbitrary degree, the generally valid smoothness term is $h^{\min (s,\ell +1)}$; extra symmetry can improve some interior biases but does not fix the general boundary counterexample.

For fixed-design nonparametric regression, let $x_i=i/n$ and $K_h(u)=h^{-1}K(u/h)$. The degree-$p$ local polynomial estimator takes the fitted intercept ${\hat{m}}_h(x;p)={\hat{\beta }}_0$, whereUse a nonnegative regression kernel and an invertible local polynomial Gram matrix to obtain a unique fit. For $p=0$, solving the single weighted least squares normal equation gives the local constant estimator:At the interior point $x_0$, use the standard unit-integral regression kernel convention ${\mu }_0(K)=1$. Otherwise the leading variance below is $R(K)/(nh{\mu }_0(K{)}^2)$; unlike a kernel density estimator, the local constant estimator itself is unchanged by rescaling $K$.

The local constant estimator is a linear estimator in nonparametric regression with weights $W_i=K_h(x_i-x_0)/(n{s}_{0,h})$. The independent unit-variance errors implyTo control the bias of an estimator, differentiability at this one point is enough. WriteSymmetry makes ${\mu }_1(K)=0$, and the supplied moment approximation gives $s_{1,h}=o(h)$. Therefore the linear contribution to the bias of an estimator is $m^{\prime }(x_0)s_{1,h}/s_{0,h}=o(h)$. The remaining contribution has absolute value at most $\eta (h)h$, because only $|x_i-x_0|\le h$ receive positive weight. This is the interior first-order bias cancellation for local constant regression, and gives squared bias of an estimator $o(h^2)$. Combining with the variance provesFor fixed $\epsilon >0$, take $h_{\epsilon }={\epsilon }^{-1/3}{n}^{-1/3}$. It satisfies both asymptotic smoothing bandwidth conditions, and the displayed remainder, multiplied by $n^{2/3}$, tends to zero for this fixed $\epsilon$. ThusLetting $\epsilon \downarrow 0$ proves the optimized scaled pointwise error tends to zero. The order of these limits matters: no remainder uniform in $\epsilon$ has been assumed.

For the lower bound over the whole function class, the absence of a uniform bound on derivatives permits a particularly direct argument. It also avoids assuming a normal distribution for errors when the model only supplies their first two moments. Fix $n$ and $x_0$. Choose a smooth bump function $b$ on the interval, with $b(x_0)=1$, whose support contains no design point except possibly $x_0$. Such a bump exists because the design is finite; at an endpoint use the restriction of a smooth bump on the real line. Every $m_a=ab$, $a\in \mathbb{R}$, belongs to the differentiability class.

If $x_0$ is not a design point, all observation means are zero for every $a$. The Le Cam two-point lemma applied to $a=\pm A$ has total variation distance zero and gives a lower bound tending to infinity with $A$. In fact, the supremum of the mean squared error is infinite at every such $x_0$.

If $x_0$ is a design point, only its response depends on $a$. The resulting location family is the single observation $Y=a+ϵ$; all remaining observations are independent of $a$ and give no extra information. The single-observation location minimax bound isHere is a proof covering discrete as well as continuous errors. Give $a$ the prior distribution $U[-A,A]$, independently of the noise. Put $p_M=\mathbb{P}(|ϵ|\le M)$ and $v_M=\mathrm{Var}(ϵ\mid |ϵ|\le M)$. Consider an easier experiment in which an oracle also reveals whether $|ϵ|\le M$, and reveals $a$ itself on the complementary event. On the event $|ϵ|\le M$ and $|Y|\le A-M$, every possible truncated noise value places $a=Y-ϵ$ within $[-A,A]$. The flat prior distribution then makes the Bayesian posterior noise law exactly its truncated original law. The smallest conditional squared-error loss is its conditional variance $v_M$, attained by the posterior mean.

The probability of that event is at least $p_M(1-2M/A)$ for $A>2M$, by restricting further to $|a|\le A-2M$. Thus every rule's supremum risk function is at least the original Bayes risk, which is at least the easier experiment's Bayes risk, and therefore at least $p_M(1-2M/A)v_M$. Let $A\to \infty$ and then $M\to \infty$. Finite second moment and zero mean give $p_M{v}_M\to 1$. The rule $T(Y)=Y$ has constant mean squared error one, proving the equality.

Consequently, at every $x_0\in [0,1]$, the unrestricted differentiability class satisfies the stronger conclusionThus $c=1$ works for every $n\ge 1$. There is no conflict with the preceding fixed-function limit: the worst functions, including arbitrarily narrow bumps, can vary with $n$. This is the pointwise versus uniform risk distinction. A smoothness ball with a common derivative bound would pose a different minimax risk problem.

Use the natural cubic spline convention appropriate to the interior design knots: the spline is linear on $[a,x_1]$ and $[x_n,b]$. We also need $n\ge 2$. These points will be important for uniqueness.

A cubic spline is a function in $C^2[a,b]$ whose restriction to each of $[a,x_1]$, $[x_1,x_2]$, through $[x_n,b]$ is a polynomial of degree at most three. A spline knot is a junction between consecutive pieces. For a natural cubic spline, the two exterior pieces are affine functions; equivalently, one uses the natural endpoint conditions $g^{\prime \prime }(x_1)=g^{\prime \prime }(x_n)=0$ on the spline over $[x_1,x_n]$ and then extends linearly with matching derivatives. Merely prescribing $g^{\prime \prime }(a)=g^{\prime \prime }(b)=0$ on otherwise arbitrary exterior cubic pieces is a different condition and does not provide the unique interpolant invoked in this question.

The natural cubic spline interpolant $N\mathbf{v}$ to values $\mathbf{v}=(v_1,\dots ,v_n{)}^T$ is the natural cubic spline satisfying $(N\mathbf{v})(x_i)=v_i$. Its existence and uniqueness for $n\ge 2$ are the interpolant fact allowed in the PDF. The interpolation map $N$ is a linear map: a linear combination of interpolants satisfies the same spline, boundary and value conditions, so uniqueness identifies it with the interpolant to the corresponding linear combination of value vectors.

Let $g=N\mathbf{v}$, let $\tilde{g}\in C^2[a,b]$ have the same values, and put $r=\tilde{g}-g$. Then $r(x_i)=0$. We prove the orthogonalityApply integration by parts twice on each polynomial interval. On an interior interval, $g^{\prime \prime \prime \prime }=0$, andThe $g^{\prime \prime \prime }r$ terms vanish because $r$ vanishes at the knots. The $g^{\prime \prime }{r}^{\prime }$ terms cancel at internal knots because $g^{\prime \prime }$ and $r^{\prime }$ are continuous, and vanish at $x_1,x_n$ because $g^{\prime \prime }$ is zero there. On the exterior intervals, $g^{\prime \prime }=0$ identically. Summing therefore gives the claimed orthogonality without assuming that $g^{\prime \prime \prime }$ is continuous across knots.

Expanding the square now gives the minimum roughness property of the natural cubic spline interpolantEquality forces $r^{\prime \prime }=0$ everywhere, since $r^{\prime \prime }$ is continuous. Thus $r$ is an affine function. It vanishes at two distinct design points, so $r=0$, proving equality if and only if $\tilde{g}=g$.

For the penalized fit, take the allowed spline roughness penalty matrix $\Gamma$ to be a symmetric matrix with ${\mathbf{z}}^T\Gamma \mathbf{z}\ge 0$; thus it is a positive semidefinite matrix. If its supplied representative were not symmetric, its symmetric part defines the same quadratic form and is all that is needed. For any $\tilde{g}$, let $\mathbf{v}=(\tilde{g}(x_i){)}_{i=1}^n$. Replacing it by $N\mathbf{v}$ preserves every fitted value and can only decrease the second derivative roughness penalty. Therefore it is enough to minimizeover ${\mathbb{R}}^n$. The matrix $A_{\lambda }=I_n+\lambda \Gamma$ is a positive-definite matrix, sinceCompleting the quadratic or differentiating gives the unique solutionThis is the cubic smoothing spline. More explicitly, the normal equations are $(I_n+\lambda \Gamma ){\hat{\mathbf{v}}}_{\lambda }=\mathbf{Y}$, andFor an arbitrary function with fitted vector $\mathbf{v}$, the earlier orthogonality also givesBoth terms are nonnegative. Equality forces $\mathbf{v}={\hat{\mathbf{v}}}_{\lambda }$ and, by the proved interpolation equality case, $\tilde{g}=N\mathbf{v}={\hat{g}}_{\lambda }$. This proves both existence in $C^2[a,b]$ and uniqueness over the entire stated function class, not just over splines.

The argument is deterministic for each observed $\mathbf{Y}$. The fixed-design nonparametric regression and homoscedasticity assumptions motivate the squared-error loss, but Gaussian errors are not required.

At least two distinct design points are required for uniqueness. Question 4 does not explicitly restate $n\ge 2$, and this hypothesis is necessary. With one design point, every function $\tilde{g}(x)=Y_1+c(x-x_1)$ has zero residual and zero second derivative roughness penalty, for any real $c$. Thus the printed unrestricted uniqueness conclusion would be false for $n=1$. The corrected theorem assumes at least two distinct knots and the stated linear-tail convention. At $\lambda =0$, uniqueness over all of $C^2[a,b]$ also fails; the stipulated $\lambda >0$ eliminates that degeneracy.

Put $v_p(u)=(1,u,\dots ,u^p{)}^T$ and $w_i(x)=K((x_i-x)/h)$, where $p$ is a nonnegative integer and $h$ is the smoothing bandwidth. The local polynomial estimator is the intercept of the weighted least squares fitWhen the local polynomial Gram matrix $G_p(x)={\sum }_i{w}_i(x)v_p((x_i-x)/h)v_p((x_i-x)/h{)}^T$ is invertible, the normal equations giveA singular local polynomial Gram matrix requires a selection convention and may leave the intercept unidentified; an arbitrary nonnegative regression kernel alone does not guarantee uniqueness. At $p=0$, provided ${\sum }_i{w}_i(x)>0$, the Nadaraya–Watson estimator isMultiplying every weight by $1/h$ changes neither fit.

For the uniform kernel, let $I_x=\{i:|i/n-x|\le h\}$ and $N_x=|I_x|$. Then ${\hat{m}}_h(x)=N_x^{-1}{\sum }_{i\in I_x}{Y}_i$. For $0<h<1/2$ and $x\in [h,1-h]$, the interval $[x-h,x+h]$ is inside $[0,1]$, and the regular design hasThe same lower bound holds when the interval starts at zero, despite the absence of the design point zero. In particular the Nadaraya–Watson estimator is defined throughout the integration interval.

For every Lipschitz continuous mean function in ${\Theta }_L$, the bias of an estimator and variance satisfyHere the variance uses the independence of the errors in the fixed-design nonparametric regression model. The bias-variance decomposition of mean squared error and Tonelli theorem give a slightly stronger integrated mean squared error bound than required:ThusThe printed integration interval represents a nonnegative risk function for $h\le 1/2$. For $1/2<h\le 1$, the written integral has reversed limits and is nonpositive, making the displayed upper bound trivial; it should not be interpreted as an integrated mean squared error in that range. For $h>1$, evaluation outside $[0,1]$ would additionally require a specified extension of the mean function. The subsequent optimization restricts to $h<1/3$.