Nonparametric statistics

Nonparametric statistics studies statistical inference without restricting the unknown probability distribution or target function to a fixed finite-dimensional parametric statistical model. In density estimation and nonparametric regression, the number of features that an estimator can represent may grow with the sample size. Rank tests offer another approach: the probability distribution of their test statistic under the null hypothesis can be obtained without specifying the common continuous probability distribution.

"Nonparametric" does not mean free of assumptions or tuning choices. A kernel density estimator still requires a kernel for density estimation and a smoothing bandwidth, and its interpretation depends on assumptions about the statistical sample. The freedom is useful when a prescribed shape, such as a single normal distribution, would hide features that the observations might reveal.

A rank test is a hypothesis test whose statistic is a function of the ranks of observations, possibly together with fixed sample labels. Under independent observations from a common continuous probability distribution, exchangeable rank allocations give its exact null law without specifying that distribution. The Wilcoxon rank sum test is an example. Invariance under common increasing transformations follows because ranks do not change.

For independent paired observations, form differences $D_i=U_i-V_i$. Assume the differences are identically distributed, continuous and symmetric about a location $\delta$. Under the zero-location null hypothesis, the signs are independent fair signs conditional on their absolute values. Rank the absolute differences and add ranks with positive signs. A negative-location alternative rejects for small $W_{+}$. Its exact conditional distribution is that of a sum of an independently selected subset of ranks, each of the $2^n$ subsets equally likely. Thus its mean is $n(n+1)/4$ and its variance is $n(n+1)(2n+1)/24$. Symmetry is essential: a zero median alone does not justify this signed-rank null law. Zero differences are omitted in the usual version, and tied absolute differences receive average ranks.

The Walsh averages of a sample are $(X_i+X_j)/2$ for $1\le i\le j\le n$, including the diagonal pairs. If the absolute observations are distinct and nonzero, order them as $|X_{(1)}|<\cdots <|X_{(n)}|$. Among the $r$ pairs whose larger absolute observation is $X_{(r)}$, every sum has the sign of $X_{(r)}$. Hence the number of positive Walsh averages is exactly ${\sum }_{r}r{1}_{X_{(r)}>0}$, the positive Wilcoxon signed-rank statistic. This proves equivalence of the corresponding tests; the Walsh averages themselves are generally dependent.

For a continuous sample, the statistic also equals ${\sum }_i{1}_{\{X_i>0\}}+{\sum }_{i<j}{1}_{\{X_i+X_j>0\}}$. In each pair the sign of the sum is the sign of the observation with larger absolute value, proving the identity by counting its smaller-rank partners. Under a continuous symmetric null, the pair kernel has projection $F(x)-1/2$ and projection variance $1/12$. The U-statistic central limit theorem yields asymptotic normality of $W_{+}$, with variance asymptotic to $n^3/12$.

For two independent samples of sizes $m,n$, rank their pooled observations and let $W_X$ be the sum of the ranks of the first sample. With continuous observations, the null hypothesis of a common distribution makes every allocation of $m$ ranks to that sample equally probable. A one-sided alternative that the first sample is stochastically smaller rejects for small $W_X$, equivalently small $U_X$. In a common-shape location model this tests a negative location difference. Without the common-shape assumption it is not simply a test of medians. The shifted statistic $U_X$ counts cross-sample pairs for which the first observation exceeds the second. Ties use average ranks and a conditional permutation test or a tie-adjusted normal approximation.

With $N=m+n$ and no ties, sampling $m$ ranks without replacement gives the displayed mean and variance. For observed rank sum below its mean, a continuity-corrected lower-tail statistic is $Z=[W_X-m(N+1)/2+1/2]/\sqrt{mn(N+1)/12}$. Ties of sizes $t_g$ replace the variance by $mn[(N+1)-{\sum }_g(t_g^3-t_g)/(N(N-1))]/12$. These formulas concern the raw rank sum; the Mann–Whitney U test statistic subtracts $m(m+1)/2$ from both the observation and its mean.

Under the continuous common-distribution null of the Wilcoxon rank sum test, pooled labels are exchangeable. The product records every choice of ranks: choosing $z{t}^k$ assigns rank $k$ to the first sample. Its indicated coefficient counts subsets of size $m$ and rank sum $w$. Divide by $\left(\genfrac{}{}{0ex}{}{m+n}{m}\right)$ and sum the lower tail for the one-sided P-value. Equivalently the counts satisfy $a_{k,h,w}=a_{k-1,h,w}+a_{k-1,h-1,w-k}$ with $a_{0,0,0}=1$, allowing an exact integer computation.

For two independent continuous samples of positive sizes $m,n$ under a common-distribution null, each $m$-subset of ranks $1,\dots ,m+n$ is equally likely. The smallest rank sum is $m(m+1)/2$, attained only by $\{1,\dots ,m\}$. The only subset with rank sum one larger is $\{1,\dots ,m-1,m+1\}$. Hence the exact lower tail at that threshold is $2/\left(\genfrac{}{}{0ex}{}{m+n}{m}\right)$. This follows directly by observing that any other change increases the sum by at least two.

Independent additive noise $Y=X+ϵ$ multiplies characteristic functions: ${\phi }_Y={\phi }_X{\phi }_{ϵ}$. When the noise characteristic function is nonzero, division recovers ${\phi }_X$. A smoothing kernel limits the frequencies used, controlling the amplification of sampling error where $|{\phi }_{ϵ}|$ is small.

Empirical likelihood places probability masses on observed or data-supported values and maximizes their product subject to normalization and any estimating constraints.

For independent event times with survivor function $S(t)=\mathbb{P}(T>t)$, an exact event at $t$ contributes mass $S(t-)-S(t)$, a right-censored observation at $c$ contributes $S(c)$, and an interval-censored observation $l<T\le u$ contributes $S(l)-S(u)$. Multiplying these factors gives the event-time likelihood when the observation mechanism is noninformative. Maximizing over nonincreasing right-continuous survivor functions gives a nonparametric maximum-likelihood estimator rather than requiring a parametric density.

Partition the time axis into cells on which membership in each observed exact-event or censoring set is constant. Let $w_j$ be a cell's probability mass and $A_{ij}$ indicate whether cell $j$ is permitted by observation $i$. The empirical likelihood has the displayed form, with nonnegative masses summing to one. Columns of $A$ that agree can be combined. If column $j$ is componentwise dominated by another column $k$, moving all mass from $j$ to $k$ cannot reduce any factor, so an unconstrained nonparametric maximum-likelihood estimator can omit column $j$. The remaining log-likelihood is a concave function of the masses. Extra constraints can prevent a domination move; for a fixed survivor function value, split cells at the constrained time and retain the associated linear mass constraint.

Suppose three exact event times $t_1<t_2<t_3$ coexist with a right-censored time $b$ and an interval $(a,c]$, ordered as $t_1<a<b<t_2<c<t_3$. Set $u=S(a)$ and $v=S(c)$. Monotonicity bounds the three event masses by $1-u$, $u-v$ and $v$, while $S(b)\le u$. Thus the mixed-censoring likelihood is bounded by the displayed expression. Equality is attained by putting all mass at $t_1,t_2,t_3$. The reduced log-likelihood is strictly concave for $0<v<u<1$, and its stationary equations yield $u=4/5$, $v=4/15$. The fitted masses are $1/5$, $8/15$, $4/15$.

A nonparametric maximum-likelihood estimator maximizes a likelihood over a nonparametric family of distributions or functions rather than over a fixed finite-dimensional parameter space.

Density estimation constructs an estimate of an unknown probability density function from sampled observations.

A Haar density estimator estimates the Haar scaling function coefficients by sample averages. It is exactly the histogram with cells of width $2^{-j}$. Its expectation is the Haar projection of the density. If the density has bounded derivative $L$ and is bounded by $B$, its pointwise expected absolute error is at most $L{2}^{-j}+\sqrt{B{2}^j/n}$: the first term is the Haar projection error for a Lipschitz function, and the second follows from the variance of a cell count and the Cauchy-Schwarz inequality. Choosing $2^j\asymp n^{1/3}$ gives error $O(n^{-1/3})$ while preserving nonnegativity and integral one.

The squared L2 norm of a probability density function derivative measures roughness and appears in optimal smoothing constants. For $r=2$, its directional derivative along a smooth compactly supported perturbation $a$ is $2\int f^{\prime \prime }{a}^{\prime \prime }$. When integration by parts is valid twice, this equals $2\int f^{(4)}a$. Thus regular root-$n$ estimation requires substantially more regularity than merely estimating a smooth probability density function.

For $A(v)=\int L^{\prime \prime }(t)L^{\prime \prime }(t+v)dt$, this U-statistic has expectation $\Vert L_b\ast f^{\prime \prime }{\Vert }_2^2$. It removes the diagonal term $\Vert L^{\prime \prime }{\Vert }_2^2/(n{b}^5)$ from the integral of a squared second derivative kernel density estimator, but leaves smoothing bias of an estimator.

In one dimension, a nearest neighbour density estimator is ${\hat{f}}_{(k)}(x)=k/[2(n+1){\rho }_{(k)}(x)]$. It uses the nearest neighbour distance to select an interval containing a fixed number of observations. The factor $n+1$ corresponds to the exact expected value of the associated uniform order statistic.

A kernel for density estimation is an integrable function $K:\mathbb{R}\to \mathbb{R}$ with unit integral. For an ordinary kernel density estimator that is itself a probability density function, takeThus $K$ specifies the bump's shape and the smoothing bandwidth $h$ specifies its width. The factor $1/h$ preserves area: doubling the smoothing bandwidth doubles the width and halves the height. An even function $K(u)=K(-u)$ is a common choice; when its absolute first moment is finite, its mean is zero. Symmetry is useful but is not required for normalization.

The Gaussian density kernel is the standard normal density, $K(u)=(2\pi {)}^{-1/2}{e}^{-u^2/2}$. Its rescaled translate $K_h(x-X_i)$ is the normal probability density with mean $X_i$ and standard deviation $h$. It is positive everywhere, whereas an Epanechnikov kernel has compact support. With the Gaussian density kernel, the smoothing bandwidth is a standard deviation, not a cutoff beyond which observations have no influence.

The broader convention also allows signed kernels of order ell, such as the flat-top kernel for density estimation, to cancel moments and reduce the bias of an estimator. Their unit integral still normalizes total area, but a resulting kernel density estimator can take negative values and need not be a probability density function. Nonnegativity is therefore an additional condition, not a consequence of unit integral.

The triweight kernel is a compactly supported symmetric probability density with variance $1/9$. Its value and first two derivatives vanish at the support endpoints. Its integrated squared second derivative is $35$, and it minimizes that curvature among twice continuously differentiable densities with zero mean and the same variance, by the triweight density minimizes integrated squared curvature inequality.

Let $f_0$ be the triweight kernel and h a twice continuously differentiable density with matching zero mean and variance. Put $e=h-f_0$ and $c=35/32$. Two integrations by parts over its support give $\int e^{\prime \prime }{f}_0^{\prime \prime }=48c[h(1)+h(-1)]+c{\int }_{|x|>1}(360x^2-72)h(x)dx\ge 0$, using the zero mass and second moment of e. Therefore $R(h^{\prime \prime })=R(f_0^{\prime \prime })+R(e^{\prime \prime })+2\int e^{\prime \prime }{f}_0^{\prime \prime }\ge R(f_0^{\prime \prime })$. The endpoint terms matter because the third derivative jumps there.

The Gaussian density kernel is the standard normal probability density used as a unit-integral kernel for density estimation. Its second moment is one and $R(K_N)=1/(2\sqrt{\pi })$. Its canonical kernel has scale $(2\sqrt{\pi }{)}^{-1/5}$. It is a function of one smoothing displacement, distinct from the two-argument Gaussian kernel used as a positive-semidefinite kernel.

The Epanechnikov kernel for density estimation is a nonnegative symmetric truncated parabola with integral one, ${\mu }_2(K_E)=1/5$, and $R(K_E)=3/5$. It has the smallest optimized asymptotic mean integrated squared error among nonnegative second-order density kernels, up to rescaling. Its canonical kernel has scale $15^{1/5}$ and support $[-15^{1/5},15^{1/5}]$.

Fix the second moment $v>0$. Put $B=\sqrt{5v}$ and $E(u)=3(B^2-u^2{)}_{+}/(4B^3)$. For every nonnegative unit-integral kernel for density estimation $K$ with the same second moment and finite $R(K)$, the mass and second-moment constraints give $\int (B^2-u^2)(K-E)=0$. Outside $[-B,B]$, $E=0$ and $(B^2-u^2)K\le 0$, so $\int E(K-E)\ge 0$. Expanding the square gives $R(K)-R(E)=\int (K-E{)}^2+2\int E(K-E)\ge 0$, with equality only when $K=E$ almost everywhere. Since $R(K{)}^2{\mu }_2(K)$ is scale invariant, rescaled Epanechnikov kernels minimize the optimized asymptotic mean integrated squared error over all such second moments. This conclusion concerns nonnegative second-order kernels; signed higher-order kernels obey a different bias expansion.

Choose a real even smooth compactly supported Fourier transform $\eta$ equal to one near zero and let $K$ be its inverse Fourier transform. Repeated integration by parts makes $K$ a Schwartz function. It is real, integrable and square-integrable, with unit integral. Moment differentiation of the Fourier transform gives $\int x^{j}K(x)dx=i^j{\eta }^{(j)}(0)=0$ for every positive integer $j$. Such a kernel for density estimation therefore satisfies arbitrary finite-order moment cancellation, and necessarily takes negative values. If the transform plateau contains the frequency support of a bandlimited function $f$, the convolution theorem proves $K\ast f=f$.

Let a bounded probability density function have Fourier transform supported in the plateau of a flat-top kernel for density estimation. At fixed smoothing bandwidth one, the convolution theorem gives $K\ast f=f$, so the kernel density estimator is pointwise unbiased. Its variance is at most $\Vert f{\Vert }_{\infty }\Vert K{\Vert }_2^2/n$, and the Cauchy-Schwarz inequality proves the displayed rate. An explicit example is $f(x)=(1-\cos x)/(\pi x^2)$, with value $1/(2\pi )$ at zero and transform $(1-|t|{)}_{+}$. Direct inverse Fourier integration gives the formula. The function is nonnegative, integrable and has integral one because its transform at zero is one. Compact frequency support also gives bounded derivatives of every order.

A kernel is of order $\ell$ when its moments of orders $1,\dots ,\ell -1$ vanish and its absolute moment of order $\ell$ is finite.

For normalized Legendre polynomials, this kernel for density estimation reproduces evaluation at zero on every polynomial of degree below $\ell$: expand $p$ in the orthonormal basis to obtain $\int K_{\ell }p=p(0)$. Taking monomials proves integral one and zero moments of orders $1,\dots ,\ell -1$; compact support gives finite absolute moments. If exact order requires a nonzero moment of order $\ell$, add $({\varphi }_{\ell }(0)+1){\varphi }_{\ell }{1}_{[-1,1]}$. Lower moments remain unchanged. Writing $a_{\ell }=⟨u^{\ell },{\varphi }_{\ell }⟩\ne 0$ and evaluating the full expansion of $u^{\ell }$ at zero shows that the new moment is $a_{\ell }$. A useful three-derivative example is $K_3=(9-15u^2)1_{[-1,1]}/8$, whose first two moments vanish and whose squared L2 norm is $9/8$.

Let $X_1,\dots ,X_n$ be real independent and identically distributed random variables with unknown probability density function $f$, let $n\ge 1$, and choose a nonnegative, unit-integral kernel for density estimation $K$ and a smoothing bandwidth $h>0$. The kernel density estimator isPlace a bump at every observation and add their contributions. Each bump $K_h(x-X_i)/n$ has area $1/n$, so for every realized statistical sampleThe change of variables is $u=(x-X_i)/h$. Summing the unweighted bumps $K_h(x-X_i)$ would give area $n$; summing just $K((x-X_i)/h)$ would give area $nh$. Those raw sums do not have the normalization of a probability density function.

Unlike a histogram, the construction does not require choosing bin origins, and a Gaussian density kernel gives a smooth curve. Its appeal is that identical local contributions can combine into a probability density function with several peaks without first choosing their number. This makes kernel density estimation useful for exploring an unknown probability distribution and estimating interval probabilities, while the smoothing bandwidth controls how much detail is visible.

Before the statistical sample is observed, ${\hat{f}}_{n,h,K}$ is a random function; afterwards it is a fitted probability density function. It is an estimator of the unknown $f$, not the true probability distribution. For fixed $K,h$, its expectation is the convolution $K_h\ast f$, generally different from $f$, and repeating the sampling changes the fitted curve. The exact mean integrated squared error of a kernel density estimator separates this smoothing bias of an estimator from sampling variance. Allowing signed kernels of order ell retains the formula but can lose nonnegativity.

Take the realized statistical sample $(-1,0,1)$, the Gaussian density kernel $\varphi (u)=(2\pi {)}^{-1/2}{e}^{-u^2/2}$ and smoothing bandwidth $h=1/2$. ThenAt $x=0$, the three contributions are $0.03599398$, $0.26596152$, and $0.03599398$, because $\varphi (0)=0.39894228$ and $\varphi (2)=e^{-2}/\sqrt{2\pi }=0.05399097$. Their sum isThis is a probability density function value, not the probability of observing exactly zero. Each of the three component curves has area $1/3$, despite having a different height at zero.

Let $\Phi$ be the cumulative distribution function of the standard normal distribution. The fitted probability of the interval $[-1/2,1/2]$ is its area under the kernel density estimator:The shaded area in the diagram is this fitted probability. Its agreement to a few digits with $1/3$ is incidental: the empirical measure gives this interval mass $1/3$ exactly, whereas kernel density estimation redistributes that mass into neighboring intervals.

Murray Rosenblatt's 1956 paper studied density estimation from a one-dimensional statistical sample, the unavoidable bias of an estimator under its stated conditions, and asymptotic mean squared error. It is an early foundation of kernel density estimation. ^[ref]. Emanuel Parzen's 1962 paper developed statistical consistency and asymptotic normality for probability density function estimates and studied estimation of the mode. ^[ref].

The authors' interests help explain the connection with smoothing. In a conversation recorded in 2006 and published in 2009, Rosenblatt recalled that his density estimation work grew out of estimation of the time-series spectral density and emphasized the connection with a smoothed histogram. This is his retrospective account of the motivation. ^[ref]. Parzen's original paper lists Stanford University as his affiliation; its opening frames the problem as recovering both a probability density function and its mode from independent and identically distributed random variables. ^[ref].

For independent observations with common density f, the kernel density estimator has mean $K_h\ast f$. Its pointwise variance is $n^{-1}[(K_h^2\ast f)-(K_h\ast f{)}^2]$. Integrate the variance and add integrated squared bias to get the displayed exact mean integrated squared error, assuming the integrals are finite. Since $R(K_h)=R(K)/h$, this also separates the smoothing bias from the stochastic variance.

For a second-order kernel density estimator, if $f(x)>0$ and $f^{\prime \prime }(x)\ne 0$, the leading pointwise asymptotic mean squared error is $R(K)f(x)/(nh)+{\mu }_2(K{)}^2{f}^{\prime \prime }(x{)}^2{h}^4/4$. Differentiation gives the displayed interior optimum. At a zero of $f$, the leading variance term vanishes and this two-term approximation has no positive interior minimizer; at a zero of $f^{\prime \prime }$, a higher-order bias expansion is needed.

For the standard normal density, $f^{\prime \prime }(x)=(x^2-1)f(x)$ and $R(f^{\prime \prime })=3/(8\sqrt{\pi })$. The fifth power of the bandwidth ratio is $3\sqrt{2}{e}^{x^2/2}/[8(x^2-1{)}^2]$. Its global minima occur at $x=\pm \sqrt{5}$; $x=0$ is only the minimum within the interval between the two inflection points. The ratio diverges as $x$ approaches either inflection point.

For a nonnegative symmetric unit-integral kernel for density estimation with $0<{\mu }_2(K)=\int u^{2}K(u)du<\infty$ and $R(K)=\int K^2<\infty$, the canonical scaling satisfies $R(K_c)={\mu }_2(K_c{)}^2$. Since $R(K_c)=R(K)/c$ and ${\mu }_2(K_c)=c^2{\mu }_2(K)$, the unique scale is $c=[R(K)/{\mu }_2(K{)}^2{]}^{1/5}$. Consequently its asymptotic mean integrated squared error is $C(K)[(nh{)}^{-1}+h^{4}R(f^{\prime \prime })/4]$, where $C(K)=R(K{)}^{4/5}{\mu }_2(K{)}^{2/5}$ is invariant under kernel rescaling. The leading optimal smoothing bandwidth becomes $[nR(f^{\prime \prime }){]}^{-1/5}$, independent of the kernel shape. Unit second moment is a different normalization and does not have this separation property.

Let the kernel for density estimation have integral one, be bounded and have compact support. For a probability density function with bounded derivative, assume $h\to 0$, $nh\to \infty$ and $n{h}^3\to 0$. The bias of a kernel density estimator is $O(h)$, negligible on the $\sqrt{nh}$ scale. For $Y_{n,i}=K((x-X_i)/h)$, $h^{-1}\mathrm{Var}{Y}_{n,i}\to f(x)\Vert K{\Vert }_2^2$. The centered summands divided by $\sqrt{nh}$ are uniformly small, so the Lindeberg-Feller central limit theorem applies. The condition $n{h}^3\to 0$ alone is insufficient: with $h=n^{-2}$ and a compactly supported kernel, with probability tending to one no observation enters the kernel window, producing a degenerate limit at a point with positive density.

Suppose a probability density function and its third derivative are bounded. A square-integrable unit-integral kernel for density estimation with first two moments zero and ${\mu }_3=\int |u{|}^3|K(u)|du<\infty$ has bias of a kernel density estimator at most $\Vert f^{(3)}{\Vert }_{\infty }{\mu }_3{h}^3/6$, by Taylor theorem with Lagrange remainder and moment cancellation. Independence gives variance at most $\Vert f{\Vert }_{\infty }\Vert K{\Vert }_2^2/(nh)$. The Cauchy-Schwarz inequality bounds the expected centred absolute deviation by the square root of this variance. Adding the bias proves the inequality uniformly in $x$. The choice $h=n^{-1/7}$ gives expected absolute error $O(n^{-3/7})$.

For an $L^2$ kernel and an independent sample from a density $f$, the centred kernel density estimator hasIndependence removes cross terms and Tonelli theorem integrates the pointwise variance. Young's convolution inequality ensures $K_h\ast f\in L^2$ even without $f\in L^2$. The Cauchy-Schwarz inequality yields the expected-norm bound $\mathbb{E}\Vert {\hat{f}}_{n,h}-K_h\ast f{\Vert }_2\le \Vert K{\Vert }_2/\sqrt{nh}$.

For a stationary autoregressive process of order one with unit-variance normal distribution marginals and fixed $0<\rho <1$, the Gaussian kernel covariance correction at lag $j$ has the displayed form. The difference of two observations has variance $2(1-{\rho }^j)$, and their characteristic functions reduce the Fourier integral to the difference of two Gaussian integrals. The mean value theorem gives $0\le g_h(j)\le \sqrt{\pi }{\rho }^j/[2(1-\rho {)}^{3/2}]$, uniformly over $h>0$. Thus the integrated variance differs from its independent-sample counterpart by $O(n^{-1})$. The marginal bias is unchanged, so this dependence does not alter the leading optimal bandwidth or the $n^{-4/5}$ mean integrated squared error for a second-order density kernel.

Over a one-dimensional Hölder density class of smoothness $\beta$, the optimized pointwise mean-squared error has order $n^{-2\beta /(2\beta +1)}$. A kernel of sufficiently high order balances squared bias $h^{2\beta }$ against variance $(nh{)}^{-1}$.

For a symmetric bandwidth-$h$ averaging kernel and a density with bounded first derivative, the interior pointwise bias is at most a constant times $h\Vert f^{\prime }{\Vert }_{\infty }$. Near a support boundary, an uncorrected symmetric kernel can instead have nonvanishing bias.

If the unknown probability distribution is supported on $[0,\infty )$, an ordinary Gaussian density kernel still puts some fitted probability below zero. For positive observations $X_i$ and the standard normal distribution cumulative distribution function $\Phi$, the displayed formula gives the total leakage. For the statistical sample $(0.1,0.4,0.9)$ and smoothing bandwidth $h=0.5$, it isThus about $22.28\%$ of the fitted probability falls outside the known support of a probability distribution, even though the full integral remains one. The leakage panel in smoothing bandwidth shades this area. Simply deleting the negative part leaves area $0.777158\dots$; renormalizing it makes another probability density function but does not by itself establish that the bias of an estimator has been corrected.

The issue also appears in the expectation of the kernel density estimator. For a nonnegative symmetric kernel for density estimation and a probability density function $f$ that is zero for $x<0$ and has a finite positive right-hand limit $f(0+)$, assume $f$ is bounded. Then, for fixed $K$ as $h\downarrow 0$,Half the averaging window lies on the impossible side of the boundary. For example, the rate-one exponential distribution has $f(0+)=1$, so the limiting expected boundary height is $1/2$, rather than the right-hand height one. This refers to estimating the right-hand limit; changing a probability density function at a single point does not change its probability distribution. Decreasing the smoothing bandwidth narrows the affected region but does not remove this boundary-height error. This is why the interior second-order pointwise bias bound for kernel density estimation cannot be applied across a jump at the boundary.

For a nonnegative symmetric unit-integral kernel for density estimation and $f\in H^2(\mathbb{R})$, the integral Taylor remainder and Minkowski inequality give $\Vert K_h\ast f-f{\Vert }_2\le h^2{\mu }_2(K)\Vert f^{\prime \prime }{\Vert }_2/2$. Translation preserves the L2 norm. For a signed kernel the same proof requires the absolute moment $\int u^2|K(u)|du$.

An uncorrected nonnegative symmetric kernel density estimator can leak an order-one amount across a jump in a probability density function. If its transition region has width proportional to the smoothing bandwidth $h$, the integrated mean squared error includes squared bias of an estimator of order $h$, rather than the order $h^4$ familiar for twice-smooth interior densities. For the unit-width uniform window and the exponential distribution of rate one, the integrated squared bias of an estimator is $h/12+o(h)$.

For a density with finitely many jumps ${\Delta }_r$ at separated points and the unit-width box kernel, take $h$ smaller than all consecutive jump spacings. The smoothing bias is supported in disjoint length-$h$ neighbourhoods of the jumps. At each jump its two sides form triangles of height $|{\Delta }_r|/2$, and exact integration givesCombined with the integrated variance of a kernel density estimator, this gives expected $L^2$ error $O(n^{-1/4})$ at $h\asymp n^{-1/2}$.

Let a nonnegative kernel for density estimation $K$ have integral one, zero first moment and finite second moment ${\mu }_2(K)$. If a probability density function $f$ has bounded second derivative, its kernel density estimator has pointwise bias at most $h^2{\mu }_2(K)\Vert f^{\prime \prime }{\Vert }_{\infty }/2$. Apply the Taylor theorem with Lagrange remainder to $f(x-hu)$ and integrate: the first-order term vanishes and the remainder is bounded by $h^2{u}^2\Vert f^{\prime \prime }{\Vert }_{\infty }/2$. For $K=1_{[-1/2,1/2]}$, ${\mu }_2(K)=1/12$.

Lepski's method compares estimators across an ordered bandwidth grid. It chooses the largest bandwidth whose confidence interval has nonempty intersection with every interval at a smaller bandwidth, thereby adapting to unknown smoothness while controlling excessive undersmoothing.

Differentiating a smooth kernel estimate givesIts integrated variance is at most $\Vert K^{\prime }{\Vert }_2^2/(n{h}^3)$.

Differentiating a twice differentiable kernel density estimator twice estimates $f^{\prime \prime }$. Its integrated variance of a kernel density estimator analogue is at most $\Vert L^{\prime \prime }{\Vert }_2^2/(n{b}^5)$. This explains why derivative estimation requires more smoothing than ordinary density estimation.

The integrated squared second derivative measures a smooth density's curvature in L2. It supplies the leading integrated squared bias constant of a second-order kernel density estimator. For nonzero second moment ${\mu }_2(K)$, the asymptotic mean integrated squared error has bias term $h^4{\mu }_2(K{)}^{2}R(f^{\prime \prime })/4$, so greater curvature calls for a smaller smoothing bandwidth.

For $f_a(x)=af(ax)$ with a positive, differentiation and change of variables give $R(f_a^{\prime \prime })=a^{5}R(f^{\prime \prime })$, while its standard deviation is $\sigma (f)/a$. Their product D is therefore invariant under scale changes, and translation also leaves it unchanged. Raw integrated squared curvature can be arbitrarily small under dilation even though this dimensionless shape measure is unchanged.

The $L^2$ Nikolsky class $\mathcal{N}(\beta ,L)$ consists of functions with $r=\lfloor \beta \rfloor$ square-integrable derivatives whose highest derivative obeys the translation boundwith the corresponding finite-difference definition at integer smoothness.

If $f^{\prime }\in \mathcal{N}(\beta ,L)$ and the kernel has sufficient order, the mean integrated squared error of ${\hat{f}}^{\prime }$ is bounded byBalancing the terms gives $h\asymp n^{-1/(2\beta +3)}$ and rate $n^{-2\beta /(2\beta +3)}$.

Nonparametric regression estimates a regression function without restricting it to a fixed finite-dimensional family.

The predictor values are random, and the regression function is the conditional mean of the response. For independent pairs and uniformly bounded conditional error variances, kernel-window occupancies govern the stochastic part of a local polynomial regression risk bound. This differs from fixed-design nonparametric regression, where the predictor values are deterministic.

A smoothing parameter controls the amount of averaging or regularization in a fitted function. Increasing the penalty multiplier $\lambda$ in a cubic smoothing spline discourages curvature and generally reduces effective degrees of freedom. Increasing a smoothing bandwidth enlarges local averaging neighbourhoods. Software may use a monotone transformed parameter such as spar, so its numerical value should not be confused with the raw penalty multiplier.

A penalized function estimate balancing a residual loss against an integrated derivative penalty. For squared-error loss and the second derivative roughness penalty, the estimate is a cubic smoothing spline. Higher derivative orders give other spline degrees.

Let $L_{i}g$ be the average of $g$ on successive knot intervals. Minimizing the displayed loss over $C^1[a,b]$ gives a quadratic spline with constant tails outside the first and last knots. To prove the reduction, choose the unique constant-tail quadratic spline $q$ with the same interval averages as a candidate $g$, and put $v=g-q$. Since $q^{\prime \prime }$ is constant on every knot interval, $q^{\prime }$ is continuous and vanishes on both tails, integration by parts gives $\int q^{\prime }{v}^{\prime }=-{\sum }_i{q}^{\prime \prime }{|}_{I_i}{\int }_{I_i}v=0$. Consequently the data loss is unchanged and $\int (g^{\prime }{)}^2=\int (q^{\prime }{)}^2+\int (v^{\prime }{)}^2$. The penalty decreases strictly unless $v$ is constant; its zero interval averages then force $v=0$. The finite-dimensional quadratic optimization on this spline space also gives a unique minimizer because the interval-average map is injective.