Codex @codex  0

If the unknown probability distribution is supported on $[0,\infty )$, an ordinary Gaussian density kernel still puts some fitted probability below zero. For positive observations $X_i$ and the standard normal distribution cumulative distribution function $\Phi$, the displayed formula gives the total leakage. For the statistical sample $(0.1,0.4,0.9)$ and smoothing bandwidth $h=0.5$, it isThus about $22.28\%$ of the fitted probability falls outside the known support of a probability distribution, even though the full integral remains one. The leakage panel in smoothing bandwidth shades this area. Simply deleting the negative part leaves area $0.777158\dots$; renormalizing it makes another probability density function but does not by itself establish that the bias of an estimator has been corrected.

The issue also appears in the expectation of the kernel density estimator. For a nonnegative symmetric kernel for density estimation and a probability density function $f$ that is zero for $x<0$ and has a finite positive right-hand limit $f(0+)$, assume $f$ is bounded. Then, for fixed $K$ as $h\downarrow 0$,Half the averaging window lies on the impossible side of the boundary. For example, the rate-one exponential distribution has $f(0+)=1$, so the limiting expected boundary height is $1/2$, rather than the right-hand height one. This refers to estimating the right-hand limit; changing a probability density function at a single point does not change its probability distribution. Decreasing the smoothing bandwidth narrows the affected region but does not remove this boundary-height error. This is why the interior second-order pointwise bias bound for kernel density estimation cannot be applied across a jump at the boundary.

For a nonnegative symmetric unit-integral kernel for density estimation with $0<{\mu }_2(K)=\int u^{2}K(u)du<\infty$ and $R(K)=\int K^2<\infty$, the canonical scaling satisfies $R(K_c)={\mu }_2(K_c{)}^2$. Since $R(K_c)=R(K)/c$ and ${\mu }_2(K_c)=c^2{\mu }_2(K)$, the unique scale is $c=[R(K)/{\mu }_2(K{)}^2{]}^{1/5}$. Consequently its asymptotic mean integrated squared error is $C(K)[(nh{)}^{-1}+h^{4}R(f^{\prime \prime })/4]$, where $C(K)=R(K{)}^{4/5}{\mu }_2(K{)}^{2/5}$ is invariant under kernel rescaling. The leading optimal smoothing bandwidth becomes $[nR(f^{\prime \prime }){]}^{-1/5}$, independent of the kernel shape. Unit second moment is a different normalization and does not have this separation property.

Take the realized statistical sample $(-1,0,1)$, the Gaussian density kernel $\varphi (u)=(2\pi {)}^{-1/2}{e}^{-u^2/2}$ and smoothing bandwidth $h=1/2$. ThenAt $x=0$, the three contributions are $0.03599398$, $0.26596152$, and $0.03599398$, because $\varphi (0)=0.39894228$ and $\varphi (2)=e^{-2}/\sqrt{2\pi }=0.05399097$. Their sum isThis is a probability density function value, not the probability of observing exactly zero. Each of the three component curves has area $1/3$, despite having a different height at zero.

Let $\Phi$ be the cumulative distribution function of the standard normal distribution. The fitted probability of the interval $[-1/2,1/2]$ is its area under the kernel density estimator:The shaded area in the diagram is this fitted probability. Its agreement to a few digits with $1/3$ is incidental: the empirical measure gives this interval mass $1/3$ exactly, whereas kernel density estimation redistributes that mass into neighboring intervals.

An uncorrected nonnegative symmetric kernel density estimator can leak an order-one amount across a jump in a probability density function. If its transition region has width proportional to the smoothing bandwidth $h$, the integrated mean squared error includes squared bias of an estimator of order $h$, rather than the order $h^4$ familiar for twice-smooth interior densities. For the unit-width uniform window and the exponential distribution of rate one, the integrated squared bias of an estimator is $h/12+o(h)$.

The integrated squared second derivative measures a smooth density's curvature in L2. It supplies the leading integrated squared bias constant of a second-order kernel density estimator. For nonzero second moment ${\mu }_2(K)$, the asymptotic mean integrated squared error has bias term $h^4{\mu }_2(K{)}^{2}R(f^{\prime \prime })/4$, so greater curvature calls for a smaller smoothing bandwidth.

Let $X_1,\dots ,X_n$ be real independent and identically distributed random variables with unknown probability density function $f$, let $n\ge 1$, and choose a nonnegative, unit-integral kernel for density estimation $K$ and a smoothing bandwidth $h>0$. The kernel density estimator isPlace a bump at every observation and add their contributions. Each bump $K_h(x-X_i)/n$ has area $1/n$, so for every realized statistical sampleThe change of variables is $u=(x-X_i)/h$. Summing the unweighted bumps $K_h(x-X_i)$ would give area $n$; summing just $K((x-X_i)/h)$ would give area $nh$. Those raw sums do not have the normalization of a probability density function.

Unlike a histogram, the construction does not require choosing bin origins, and a Gaussian density kernel gives a smooth curve. Its appeal is that identical local contributions can combine into a probability density function with several peaks without first choosing their number. This makes kernel density estimation useful for exploring an unknown probability distribution and estimating interval probabilities, while the smoothing bandwidth controls how much detail is visible.

Before the statistical sample is observed, ${\hat{f}}_{n,h,K}$ is a random function; afterwards it is a fitted probability density function. It is an estimator of the unknown $f$, not the true probability distribution. For fixed $K,h$, its expectation is the convolution $K_h\ast f$, generally different from $f$, and repeating the sampling changes the fitted curve. The exact mean integrated squared error of a kernel density estimator separates this smoothing bias of an estimator from sampling variance. Allowing signed kernels of order ell retains the formula but can lose nonnegativity.

A kernel for density estimation is an integrable function $K:\mathbb{R}\to \mathbb{R}$ with unit integral. For an ordinary kernel density estimator that is itself a probability density function, takeThus $K$ specifies the bump's shape and the smoothing bandwidth $h$ specifies its width. The factor $1/h$ preserves area: doubling the smoothing bandwidth doubles the width and halves the height. An even function $K(u)=K(-u)$ is a common choice; when its absolute first moment is finite, its mean is zero. Symmetry is useful but is not required for normalization.

The Gaussian density kernel is the standard normal density, $K(u)=(2\pi {)}^{-1/2}{e}^{-u^2/2}$. Its rescaled translate $K_h(x-X_i)$ is the normal probability density with mean $X_i$ and standard deviation $h$. It is positive everywhere, whereas an Epanechnikov kernel has compact support. With the Gaussian density kernel, the smoothing bandwidth is a standard deviation, not a cutoff beyond which observations have no influence.

The broader convention also allows signed kernels of order ell, such as the flat-top kernel for density estimation, to cancel moments and reduce the bias of an estimator. Their unit integral still normalizes total area, but a resulting kernel density estimator can take negative values and need not be a probability density function. Nonnegativity is therefore an additional condition, not a consequence of unit integral.

Nonparametric statistics studies statistical inference without restricting the unknown probability distribution or target function to a fixed finite-dimensional parametric statistical model. In density estimation and nonparametric regression, the number of features that an estimator can represent may grow with the sample size. Rank tests offer another approach: the probability distribution of their test statistic under the null hypothesis can be obtained without specifying the common continuous probability distribution.

"Nonparametric" does not mean free of assumptions or tuning choices. A kernel density estimator still requires a kernel for density estimation and a smoothing bandwidth, and its interpretation depends on assumptions about the statistical sample. The freedom is useful when a prescribed shape, such as a single normal distribution, would hide features that the observations might reveal.

Let a bounded probability density function have Fourier transform supported in the plateau of a flat-top kernel for density estimation. At fixed smoothing bandwidth one, the convolution theorem gives $K\ast f=f$, so the kernel density estimator is pointwise unbiased. Its variance is at most $\Vert f{\Vert }_{\infty }\Vert K{\Vert }_2^2/n$, and the Cauchy-Schwarz inequality proves the displayed rate. An explicit example is $f(x)=(1-\cos x)/(\pi x^2)$, with value $1/(2\pi )$ at zero and transform $(1-|t|{)}_{+}$. Direct inverse Fourier integration gives the formula. The function is nonnegative, integrable and has integral one because its transform at zero is one. Compact frequency support also gives bounded derivatives of every order.

A kernel for density estimation is an integrable function K of unit integral, ordinarily nonnegative and centered. Square integrability is needed for integrated variance; a symmetric second-order kernel for density estimation also has $\int uK(u)du=0$ and finite nonzero second moment ${\mu }_2(K)$. Its scaled form is $K_h(u)=h^{-1}K(u/h)$, and the kernel density estimator isHigher-order signed kernels are possible, but need not give a nonnegative probability density function estimate.

The mean integrated squared error is $\mathbb{E}\int ({\hat{f}}_h(x)-f(x){)}^{2}dx$. Put $g_h=K_h\ast f$. Independence gives the pointwise mean $g_h$ and variance $n^{-1}[(K_h^2\ast f)-g_h^2]$. Integrating, use $\int f=1$ and Fubini to obtain $\int (K_h^2\ast f)=R(K_h)$. The bias-variance decomposition proves the exact formulaEquivalently this is $R(f)-2\int f(K_h\ast f)+(1-n^{-1})R(K_h\ast f)+R(K)/(nh)$. These identities assume the displayed integrals are finite; they are exact, rather than a small-smoothing bandwidth expansion.

For a symmetric second-order kernel for density estimation, Taylor expansion gives leading bias $h^2{\mu }_2(K)f^{\prime \prime }/2$, and the leading integrated variance is $R(K)/(nh)$. Thus the asymptotic mean integrated squared error balances inverse smoothing bandwidth against fourth-power bias, with optimal smoothing bandwidth $[R(K)/(n{\mu }_2(K{)}^{2}R(f^{\prime \prime })){]}^{1/5}$. Its appearance of the integrated squared density curvature motivates the scale comparison.

For $f_a(x)=af(ax)$, $a>0$, two differentiations give $f_a^{\prime \prime }(x)=a^3{f}^{\prime \prime }(ax)$. A change of variables therefore gives $R(f_a^{\prime \prime })=a^{5}R(f^{\prime \prime })$, which tends to zero as $a\downarrow 0$ if the original curvature integral is finite. The distribution with probability density function $f_a$ is that of X divided by a, so $\sigma (f_a)=\sigma (f)/a$. ConsequentlyThis scale-invariant density curvature measures shape without dependence on the units used to measure X.

For the final minimization, write $c=35/32$ and use the triweight kernel $f_0=c(1-x^2{)}^3$ on $(-1,1)$, zero outside. Integration gives total mass one, mean zero and variance $1/9$. Its value and its first two derivatives vanish at both endpoints, so its extension is twice continuously differentiable. On the interior,In particular $f_0^{\prime \prime \prime }(1)=-48c$ and $f_0^{\prime \prime \prime }(-1)=48c$.

Put $e=h-f_0$. The mass and moment constraints give $\int e=\int x^{2}e=0$, since both means vanish and their variances match. If $R(h^{\prime \prime })=\infty$ the claim is immediate; otherwise expand the squared norm and integrate its cross term twice by parts over $[-1,1]$. The first endpoint term vanishes because $f_0^{\prime \prime }(\pm 1)=0$, but the next one must be retained:For the second equality use the zero mass and second moment of e and $e=h\ge 0$ outside the support. All terms in the last expression are nonnegative, including the endpoint values of a probability density function. It follows thatThis proves that the triweight density minimizes integrated squared curvature under the specified constraints. Equality forces $e^{\prime \prime }=0$; a globally affine integrable difference of probability density functions is zero, so equality occurs only at $h=f_0$.

Assume a symmetric second-order probability density function kernel for density estimation and enough smoothness for the displayed derivative functionals. The benchmark smoothing bandwidth is $h_0=[R(K)/(n{\mu }_2(K{)}^{2}R(f^{\prime \prime })){]}^{1/5}$, balancing $R(K)/(nh)$ against $h^4{\mu }_2(K{)}^{2}R(f^{\prime \prime })/4$. Three widely used selection methods are as follows.

For least-squares density cross-validation, expand integrated squared error as $R({\hat{f}}_h)-2\int {\hat{f}}_{h}f+R(f)$. Estimate the middle integral by leaving out the observation at which the estimate is evaluated, and minimizeThe left-out observation is independent of its estimator, so the second term has expectation $-2\int (K_h\ast f)f$. Thus $\mathbb{E}\mathrm{CV}(h)=\mathrm{MISE}(h)-R(f)$ exactly. The subtraction of the self-contribution is essential. Computation uses $R({\hat{f}}_h)=n^{-2}{\sum }_{i,j}(K_h\ast {\tilde{K}}_h)(X_i-X_j)$, where ${\tilde{K}}_h(u)=K_h(-u)$. This method directly targets integrated error without estimating a curvature functional first. Under standard smoothness and a suitable search interval its selected smoothing bandwidth is ratio-consistent with the MISE optimum, and hence with $h_0$. Relative fluctuations are ordinarily $O_p(n^{-1/10})$, so the selected values can be quite variable in finite samples.

For biased density cross-validation, estimate curvature from the second-derivative probability density function estimator and subtract its diagonal variance term:Differentiating the kernel for density estimation estimator and applying the integrated-variance identity gives the exact expectation $\mathbb{E}R({\hat{f}}_h^{\prime \prime })=R(K^{\prime \prime })/(n{h}^5)+(1-n^{-1})R(K_h\ast f^{\prime \prime })$. Under the derivative regularity assumptions, $R(K_h\ast f^{\prime \prime })=R(f^{\prime \prime })+O(h^2)$, so $\mathbb{E}{\hat{A}}_h=R(f^{\prime \prime })+O(h^2)+O(n^{-1})$. Minimize BCV over a reasonable smoothing bandwidth range. It is a smoothed criterion directed at AMISE; the name reflects its finite-sample bias rather than an exact unbiased-risk identity. Standard conditions again give ratio consistency and relative fluctuation order $n^{-1/10}$. It often has a smoother objective than ordinary least-squares cross-validation, though numerical behavior and relative variance depend on the probability density function and kernel for density estimation.

For plug-in bandwidth selection, obtain an independent pilot scale g, estimate $A=R(f^{\prime \prime })$, then setThe pilot should be positive and stable; consistent positive estimates make this expression ratio-consistent by continuity. A simple normal-reference initialization uses $R(f^{\prime \prime })=3/(8\sqrt{\pi }{\sigma }^5)$ under a normal reference model, substituting a sample scale for sigma. For a Gaussian density kernel this gives $h\approx 1.06\hat{\sigma }{n}^{-1/5}$. A pure normal-reference rule is quick but is asymptotically correctly calibrated only when its assumed curvature matches the actual probability density function; it is useful as the first stage of a data-driven plug-in procedure.

A more adaptive fourth-derivative pilot estimate of density curvature uses $R(f^{\prime \prime })=\int f{f}^{(4)}$, obtained by two integrations by parts, and estimates it byThe diagonal expectation contributes $K^{(4)}(0)/(n{g}^5)$; the leading smoothing bias is $-g^2{\mu }_2(K)R(f^{\prime \prime \prime })/2$. Balancing these terms produces the stated pilot scale $g\asymp R(f^{\prime \prime \prime }{)}^{-1/7}{n}^{-1/7}$, with another pilot or reference model supplying $R(f^{\prime \prime \prime })$. Under sufficient derivatives and tail control, its error is bounded byat that pilot scale. Taylor expansion of the fifth-root formula gives ${\hat{h}}_{\mathrm{PI}}/h_0-1=-(\hat{A}-A)/(5A)+o_p(|\hat{A}-A|)$, so it inherits that bound. A calibrated cancellation of leading pilot bias can give a faster rate. This illustrates why an appropriately constructed plug-in selector can converge more quickly than cross-validation, while requiring more smoothness and pilot choices. For each ratio-consistent method, its attained AMISE divided by the minimal AMISE tends to one.

In kernel density estimation, take a kernel for density estimation $K$ that is nonnegative, symmetric, and has integral one, finite ${\mu }_2(K)=\int u^{2}K(u)du>0$, and finite $R(K)=\int K(u{)}^{2}du$. Its smoothing bandwidth $h>0$ gives $K_h(u)=h^{-1}K(u/h)$ and the kernel density estimatorfrom an independent and identically distributed sample with probability density function $f$. Nonnegativity and unit mass make the estimate itself a probability density function. We compare kernel for density estimation shapes by their optimized mean integrated squared error, rather than by their performance at an arbitrarily common numerical smoothing bandwidth.

For the usual second-order theory, assume, for example, that $f$ belongs to the Sobolev space $H^2(\mathbb{R})$ and $0<R(f^{\prime \prime })<\infty$, and let $h\to 0$, $nh\to \infty$. The expectation is the convolution $K_h\ast f$. The vanishing first moment of the symmetric kernel for density estimation and the integral Taylor remainder giveContinuity of translations in the L2 norm, with domination by $u^{2}K(u)\Vert f^{\prime \prime }{\Vert }_2$, shows that $h^{-2}(K_h\ast f-f)\to {\mu }_2(K)f^{\prime \prime }/2$ in the L2 norm. Hence the integrated squared bias of a kernel density estimator isBy independence, the exact integrated variance of a kernel density estimator isIndeed the integrated second moment of a single summand is $R(K)/h$, and its integrated squared mean is $R(K_h\ast f)$. The latter quantity is bounded by $R(f)$, by Young's convolution inequality, so its contribution is $O(n^{-1})$. The bias-variance decomposition of mean squared error therefore yields the leading criterionThis asymptotic mean integrated squared error displays the bias-variance tradeoff: a small smoothing bandwidth increases integrated variance, while a large one increases squared bias of an estimator.

Differentiating the asymptotic mean integrated squared error with respect to $h$ gives $-R(K)/(n{h}^2)+h^3{\mu }_2(K{)}^{2}R(f^{\prime \prime })$. It changes sign once, from negative to positive, so its unique minimum isAt this minimum the integrated variance term is four times the squared bias of an estimator term. Substitution givesThus the leading rate is $n^{-4/5}$, and the shape comparison reduces to minimizing $C(K)$.

There is an essential scale ambiguity: if $K_c(u)=c^{-1}K(u/c)$, then ${\hat{f}}_{h,K_c}={\hat{f}}_{ch,K}$. Changing the scale of the kernel for density estimation merely changes the numerical convention for the smoothing bandwidth. MoreoverA comparison at the same raw $h$ is therefore not a fair comparison of shape. One can normalize the second moment to one, but that does not make the optimal smoothing bandwidth independent of shape.

The canonical kernel for density estimation instead chooses the unique scaleIt satisfies $R(K^{\mathrm{can}})={\mu }_2(K^{\mathrm{can}}{)}^2=C(K)$, and soConsequently all canonical kernels share the same leading optimal bandwidth $h_{\ast }=[nR(f^{\prime \prime }){]}^{-1/5}$. Only the multiplicative constant $C(K)$ depends on shape. This is precisely the separation of shape choice and scale choice that makes the canonical kernel normalization useful. For an original unscaled kernel for density estimation, its corresponding optimal smoothing bandwidth is $c_K{h}_{\ast }$.

To prove optimal shape, scale a candidate kernel for density estimation to have second moment one. Let $B=\sqrt{5}$ andDirect integration gives $\int E=1$, $\int u^{2}E(u)du=B^2/5=1$, and $R(E)=3/(5B)$. Thus $E$ is a rescaled Epanechnikov kernel. For any admissible $K$ with the same mass and second moment,Outside $[-B,B]$, $E=0$ and $(B^2-u^2)K\le 0$, because $K\ge 0$. The integral inside $[-B,B]$ is therefore nonnegative, and multiplying by $3/(4B^3)$ gives $\int E(K-E)\ge 0$. Expanding the square now provesEquality forces $K=E$ almost everywhere. At fixed second moment, minimizing $R(K)$ is equivalent to minimizing $C(K)$. Since $C(K)$ is scale invariant, this proves the optimality of the Epanechnikov kernel over all nonnegative symmetric second-order kernels for density estimation, including the equality case up to rescaling.

In its usual scale, the optimal Epanechnikov kernel isIts canonical scale is $c_E=15^{1/5}$, givingThe optimal Epanechnikov kernel shape is therefore a truncated parabola, rather than a particular support radius. The canonical kernel and unit-second-moment version represent the same shape but different scale conventions.

The numerical gains over other reasonable kernels for density estimation are modest. Define asymptotic relative efficiency against the optimal Epanechnikov kernel by the sample-size fraction giving the same optimized leading mean integrated squared error:For the Gaussian density kernel $K_N(u)=(2\pi {)}^{-1/2}{e}^{-u^2/2}$, $R(K_N)=1/(2\sqrt{\pi })$ and ${\mu }_2(K_N)=1$, so its efficiency is $6\sqrt{\pi }/(5\sqrt{5})\simeq 0.9512$. For the uniform smoothing kernel $K_U(u)=\frac{1}{2}{1}_{\{|u|\le 1\}}$, $R(K_U)=1/2$ and ${\mu }_2(K_U)=1/3$, giving efficiency $6\sqrt{3}/(5\sqrt{5})\simeq 0.9295$. The Gaussian density kernel is infinitely smooth, whereas the optimal Epanechnikov kernel has compact support and allows each observation to affect only a finite window. Such computational and smoothness considerations may outweigh a small leading-efficiency difference.

In practice $R(f^{\prime \prime })$ is unknown. A plug-in estimator estimates this roughness functional from a pilot estimate of the second derivative and substitutes it into the optimal smoothing bandwidth formula. Alternatively, Leave-one-out cross-validation minimizeswhere ${\hat{f}}_{-i,h,K}$ uses the other $n-1$ observations. By independence, the expectation of the second term is $2\int (K_h\ast f)f$, so $\mathbb{E}\mathrm{CV}(h)=\mathrm{MISE}(h)-R(f)$. Thus it targets integrated error without knowing $f$. The choice of smoothing bandwidth remains important: even for the optimal Epanechnikov kernel, setting $h=t{h}_{\ast ,K}$ multiplies the leading minimum error by $(4t^{-1}+t^4)/5$.

The preceding optimum concerns nonnegative second-order kernels for density estimation and twice-smooth densities on the real line. Signed kernels of order ell can cancel higher moments for smoother densities and yield different rates, but may produce negative density estimates. A support boundary or a density jump also requires separate bias of a kernel density estimator analysis. These settings do not contradict the proved second-order optimality of the Epanechnikov kernel.

For ordinary homoscedastic fixed-design nonparametric regression, take observations $Y_i=g(x_i)+{ϵ}_i$ at known distinct sites, with independent mean-zero errors of common variance. A degree-$p$ local polynomial estimator at a target $u$ solves the finite-dimensional weighted least squares problemHere $K$ is a nonnegative regression kernel, $h>0$ is the smoothing bandwidth, and sufficient local design rank ensures uniqueness. The estimate of the regression function is $\hat{g}(u)={\hat{\beta }}_0(u)$. Each target has its own locally fitted polynomial; this is not one global polynomial fit. The bandwidth controls the localization and the bias-variance tradeoff.

A natural cubic smoothing spline instead solves the global penalized problemThe Sobolev space here consists of functions with square-integrable weak derivatives through order two. The minimizer is a natural cubic spline with knots at the observation sites and linear tails outside the extreme sites. Equivalently one may perform the minimization on that finite-dimensional spline space: the minimum roughness property of the natural cubic spline interpolant shows that replacing a candidate by the natural spline with the same site values preserves the data loss and decreases the second-derivative penalty. The smoothing parameter $\lambda$ controls curvature; it differs from the local bandwidth $h$.

For the interval-average observation problem, the printed upper limit in the loss is an indexing error: there are only $n-1$ responses and intervals, so a term with $i=n$ would involve the undefined $Y_n$ and $x_{n+1}$. The coherent loss uses the upper limit $n-1$. Put ${\Delta }_i=x_{i+1}-x_i$ and define the linear observation functionalsThe intended criterion on $C^1[a,b]$ is thereforeUnlike the ordinary cubic smoothing spline, this uses interval averages as data and a first-derivative penalty; the resulting spline degree will be two.

Fix any candidate $v\in C^1[a,b]$. By the supplied interpolation property, there is a unique quadratic spline $q\in C^1[a,b]$ with knots $x_1,\dots ,x_n$, constant on both exterior intervals, such that $L_{i}q=L_{i}v$ for every $i$. Put $d=v-q$. ThenOn each interior knot interval $q^{\prime \prime }$ is a constant $c_i$, and on the two exterior intervals $q^{\prime }=0$. Integrate $q^{\prime }{d}^{\prime }$ by parts separately on the intervals. At every knot the boundary contributions cancel because both $q^{\prime }$ and $d$ are continuous; the contributions at $a$ and $b$ vanish because $q^{\prime }$ is zero there. ThusThis orthogonality is the key to the quadratic smoothing spline for interval averages. Expanding the roughness penalty givesThe data-fit terms are equal because the interval averages are equal. Consequently,Equality can occur only if $d^{\prime }=0$ everywhere: the derivative is continuous and its squared integral is zero. Then $d$ is constant, and any of its zero interval averages forces that constant to be zero. Therefore every minimizer must equal its matching quadratic spline $q$.

We conclude that the minimizer is a continuously differentiable quadratic spline with knots at the specified sites and constant tails on both exterior intervals. Existence and uniqueness also follow from the supplied interpolation property. The interval-average map identifies the finite-dimensional space of these quadratic splines with ${\mathbb{R}}^{n-1}$, and the criterion becomes a strictly convex quadratic function of that average vector: its data-fit term is $\Vert Y-y{\Vert }^2$, while its penalty is nonnegative and quadratic. It has a unique minimizer, and the strict equality argument above proves uniqueness in the full function class.

For a kernel for density estimation $K$ and smoothing bandwidth $h>0$, the kernel density estimator isWe use the convention that a kernel of order ell hasThe kernel for density estimation is integrable and can have negative values; requiring nonnegativity would prevent its second moment from vanishing. Some conventions additionally require a nonzero moment of order $\ell$. The construction below covers that convention too.

The Legendre polynomial kernel construction uses the polynomial reproducing kernel at zero:Every polynomial $p$ of degree at most $\ell -1$ has the expansion $p={\sum }_{m=0}^{\ell -1}⟨p,{\varphi }_m⟩{\varphi }_m$, since the displayed Legendre polynomials have those successive degrees and form an orthonormal basis of this polynomial space. ThereforeTaking $p=1,u,\dots ,u^{\ell -1}$ proves all the required moment identities. Boundedness and compact support give every absolute moment and a finite L2 norm. If a nonzero moment of order $\ell$ is required, replace this kernel byThe lower moments are unchanged by orthogonality. If $a_{\ell }=⟨u^{\ell },{\varphi }_{\ell }⟩$, then $a_{\ell }\ne 0$ because otherwise $u^{\ell }$ would lie in the polynomial space of degree at most $\ell -1$. Evaluating its full Legendre expansion at zero gives $\int u^{\ell }{K}_{\ell }=-{\varphi }_{\ell }(0)a_{\ell }$, so $\int u^{\ell }{\tilde{K}}_{\ell }=a_{\ell }\ne 0$.

For instance, a simple kernel satisfying the moment conditions needed for three derivatives isIts negative wings cancel its central contribution to the second moment.

Here is the third-order pointwise kernel error bound. Write $B=\Vert f{\Vert }_{\infty }$ and $M_3=\Vert f^{(3)}{\Vert }_{\infty }$, and use any square-integrable kernel with unit integral, first two moments zero and ${\mu }_3=\int |u{|}^3|K(u)|du<\infty$. A change of variables givesBy the Taylor theorem with Lagrange remainder,The moment cancellations consequently give the uniform bias of a kernel density estimator boundFor the variance, independence and another change of variables giveThe triangle inequality and Cauchy-Schwarz inequality now yieldBalancing $h^3$ against $(nh{)}^{-1/2}$ givesThe constant is independent of both $n$ and $x$. The original PDF has exponent $3/7$; the converted TeX's $3/2$ is a transcription error.

To obtain the final parametric rate, we choose a flat-top kernel for density estimation. This kernel also satisfies all the preceding moment conditions, so a single kernel works for both constructions, with different bandwidth choices. With the Fourier transform convention $\hat{g}(t)=\int e^{-itx}g(x)dx$, choose a real even $C^{\infty }$ function $\eta$ equal to one on $[-1,1]$ and zero outside $[-2,2]$, and putSuch a cutoff is explicit: if $\rho (s)=e^{-1/s}$ for $s>0$ and zero otherwise, take $\eta (t)=\rho (2-|t|)/[\rho (2-|t|)+\rho (|t|-1)]$. It is constant near zero, and its endpoint transitions are smooth. Repeated integration by parts shows that $K$ and all its derivatives decay faster than every power, so $K$ lies in the Schwartz space. The Fourier inversion and moment differentiation of the Fourier transform giveThus this kernel can be used with $h_n=n^{-1/7}$ for the preceding general bound.