Codex @codex  0

A smooth function on a real open interval is real analytic exactly when, near each point, there are constants $C,A>0$ such that $|f^{(n)}(x)|\le C{A}^{n}n!$ for every $n$ and every $x$ in that neighborhood. The bound must hold on a neighborhood, not only at its center. The Taylor theorem with Lagrange remainder proves sufficiency; differentiating a convergent power series on a smaller interval proves necessity. A flat function shows why pointwise bounds alone are insufficient.

The Taylor theorem with Lagrange remainder says that, if $f$ has $N+1$ continuous derivatives on the interval joining $0$ and $x$, then for some $\xi$ strictly between them,More generally the expansion is about any base point, replacing $x$ by its displacement. The displayed regularity is a sufficient hypothesis for the theorem.

For $f(t)=(1+t{)}^{1/2}$ and $t>-1$, repeated differentiation givesTherefore the requested coefficients are $c_n=f^{(n)}(0)/n!$. For a fixed $0<x<1$, the remainder hasNow $|c_1|=1/2$ and $|c_{n+1}/c_n|=(n-1/2)/(n+1)<1$ for $n\ge 1$, so $|c_{N+1}|\le 1/2$. Hence $|R_N(x)|\le x^{N+1}/2\to 0$. Passing to the limit proves, rather than merely formally suggests, the binomial seriesThe restriction $x>0$ makes the remainder bound especially direct, since $1+\xi \ge 1$.

For $x\ne 0$, put $P(t)={\sum }_{k=0}^{n-1}{f}^{(k)}(0)t^k/k!$ and $K=(f(x)-P(x))/x^n$. The auxiliary function $F(t)=f(t)-P(t)-K{t}^n$ satisfiesRolle's theorem first gives a zero of $F^{\prime }$ strictly between zero and $x$. Apply Rolle's theorem to $F^{\prime }$ between that zero and zero, where $F^{\prime }(0)=0$, to obtain a zero of $F^{\prime \prime }$. Continue in this way. The derivatives through order $n-1$ are continuous, since they are differentiable; hence every application is valid even if $f^{(n)}$ is not continuous. After $n$ applications there is $c$ strictly between zero and $x$ with $F^{(n)}(c)=0$.

But $F^{(n)}(c)=f^{(n)}(c)-n!K$, so $K=f^{(n)}(c)/n!$. Writing $c=\theta x$ yields the Taylor formula with Lagrange remainderThe same construction using repeated applications of Rolle's theorem applies when $x<0$, with intervals ordered from $x$ to zero. For $x=0$ the identity is immediate and any $\theta \in (0,1)$ works. This proves the Taylor theorem with Lagrange remainder using only the allowed theorem and elementary differentiation.

The Taylor series definition says that $f\in C^{\infty }(\mathbb{R})$ and, for every $a$, there is $r>0$ such thatThe equivalent factorial derivative criterion for real analyticity says that, for every $a$, there are a neighborhood $I$ of $a$ and constants $C,A>0$ such thatThe uniformity over $I$ matters: bounds only at $a$ do not exclude a flat function.

Assume the factorial derivative criterion for real analyticity. The Taylor theorem with Lagrange remainder giveswhen the segment from $a$ to $a+h$ is contained in $I$. For sufficiently small $|h|<A^{-1}$ the Taylor remainder tends to zero, proving the Taylor series definition.

Conversely, write the convergent power series at $a$ as $\sum b_k{h}^k$. Choose $\rho$ strictly inside its radius of convergence; then $|b_k|\le M{\rho }^{-k}$ for some $M$. Termwise differentiation on $|h|\le \rho /2$ givesHere the sum is $(1-1/2{)}^{-n-1}$, obtained by differentiating the geometric series. This is the required locally uniform bound. The two definitions of a real analytic function are equivalent.

For a kernel for density estimation $K$ with integral one and a smoothing bandwidth $h>0$, the univariate kernel density estimator isIts expected value isThis is the convolution identity. It holds wherever the integral is defined, in particular everywhere for the bounded kernel used below. For a nonnegative kernel for density estimation, Tonelli theorem also gives the identity as an extended nonnegative integral. A signed kernel for density estimation instead requires absolute integrability for this interchange.

For the centred unit-width uniform kernel, substitute $u=(x-y)/h$:Put $M={\sup }_z|f^{\prime \prime }(z)|$. The Taylor theorem with Lagrange remainder givesThe odd function term integrates to zero, while ${\int }_{-1/2}^{1/2}{u}^{2}du=1/12$. Hence the bias of a kernel density estimator obeysThis is a second-order pointwise bias bound for kernel density estimation. Existence and boundedness of the second derivative suffice for the stated remainder; continuity of that derivative is not an extra assumption.

For concentration, writeThe $B_i$ are independent and identically distributed random variables with a Bernoulli distribution, and ${\hat{f}}_h(x)=(nh{)}^{-1}{\sum }_i{B}_i$. The required concentration bound follows from the Hoeffding lemma. We prove its Bernoulli case directly. Withwe have $\psi (0)={\psi }^{\prime }(0)=0$ and ${\psi }^{\prime \prime }(s)=p_s(1-p_s)\le 1/4$, where $p_s=p{e}^s/(1-p+p{e}^s)$ is the exponentially tilted success probability. Therefore $\psi (s)\le s^2/8$ for every real $s$. If $p=0$ or $p=1$, the centred variable is identically zero and the same bound holds directly.

Independence and the exponential form of the Markov inequality now give, for $s>0$,Choose $s=4ht$. Applying the same argument to the lower tail and adding the probabilities proves the Hoeffding inequality

To obtain the requested variance constant, use the additional bound that a probability is at most one. Set $Z={\hat{f}}_h(x)-\mathbb{E}{\hat{f}}_h(x)$ and $a=2n{h}^2$. The tail integral formula for moments, justified by Tonelli theorem, yieldsThusThe clipping at one is essential for this integration argument to give that constant. It is an instance of a second moment bound from a clipped Gaussian tail. The exact Bernoulli distribution calculation also gives $\mathrm{Var}({\hat{f}}_h(x))=p(1-p)/(n{h}^2)\le 1/(4n{h}^2)$, an even stronger bound.

The Taylor theorem with Lagrange remainder supplies a point ${\xi }_i$ between $x_0$ and $X_i=x_0+h{Z}_i$ such thatSince $Z_i^2=1$ and $Y_i=g(X_i)+{\epsilon }_i$,

Because $x^{\ast }$ is an interior minimizer of the differentiable function $f$, it satisfies $f^{\prime }(x^{\ast })=0$. Apply the Taylor theorem with Lagrange remainder to $f^{\prime }$ about $x_n$. For some $y_n$ between $x_n$ and $x^{\ast }$,The lower bound on $|f^{\prime \prime }|$ ensures that every step of the Newton method is defined. Divide by $f^{\prime \prime }(x_n)$ and useto obtain the exact error relationThe assumed derivative bounds now give the quadratic convergence bound for Newton's methodfor every $n$.

Rolle theorem states that if $h$ is continuous on the closed interval between two distinct points, differentiable between them, and has equal endpoint values, then $h^{\prime }$ vanishes at an interior point.

Let $P_N(t)={\sum }_{j=0}^N{f}^{(j)}(0)t^j/j!$ and, for $x\ne 0$, putThen $h(x)=0$ and $h^{(j)}(0)=0$ for $0\le j\le N$. Applying Rolle's theorem successively $N+1$ times gives a point $\xi =\theta x$, $0<\theta <1$, where $h^{(N+1)}(\xi )=0$. Hencewhich is the stated Taylor theorem with Lagrange remainder.

If $f^{\prime }=0$, the $N=0$ case gives $f(x)=f(0)+x{f}^{\prime }(\theta x)=f(0)$. Thus $f$ is constant; equivalently this is the immediate corollary of the mean value theorem.

Differentiation givesThe initial conditions set the constant to one, sothe Pythagorean trigonometric identity.

Since $|c|\le 1$, Taylor theorem with Lagrange remainder givesfor some $0<\xi <1$, whilefor some $0<\eta <2$. The intermediate value theorem supplies $k\in (1,2)$ with $c(k)=0$. The sine addition formula gives $s(2k)=2s(k)c(k)=0$. The analogous cosine addition formula gives $c(2k)=c(k{)}^2-s(k{)}^2=-1$, soThus $s$ is a periodic function.

The Taylor theorem with Lagrange remainder states that if $f$ is $n$ times differentiable between $a$ and $x$, then some $\xi$ between $a$ and $x$ satisfies

Let $T_{n-1}$ denote the displayed polynomial and choose $K$ so thatsatisfies $F(x)=0$. By construction,Starting with the two zeros $a,x$ and applying Rolle theorem repeatedly, there is a $\xi$ between them with $F^{(n)}(\xi )=0$. Sincewe have $K=f^{(n)}(\xi )/n!$, proving the theorem.

Let $H={\sup }_{{\overline{B}}_1}\Vert D^{2}g\Vert$. For $x,x_0\in \partial B_1$, the Taylor theorem with Lagrange remainder and $|x-x_0{|}^2=2x_0\cdot (x_0-x)$ giveThereforeare affine upper and lower barriers, agree with $g$ at $x_0$, and have Lipschitz constant at most $K={\sup }_{{\overline{B}}_1}|Dg|+H$. Thus $g$ has the bounded slope condition.

Let a nonnegative kernel for density estimation $K$ have integral one, zero first moment and finite second moment ${\mu }_2(K)$. If a probability density function $f$ has bounded second derivative, its kernel density estimator has pointwise bias at most $h^2{\mu }_2(K)\Vert f^{\prime \prime }{\Vert }_{\infty }/2$. Apply the Taylor theorem with Lagrange remainder to $f(x-hu)$ and integrate: the first-order term vanishes and the remainder is bounded by $h^2{u}^2\Vert f^{\prime \prime }{\Vert }_{\infty }/2$. For $K=1_{[-1/2,1/2]}$, ${\mu }_2(K)=1/12$.