Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 210 (Topics in Statistical Theory)

Writing $P_{n}g=n^{-1}{\sum }_{i=1}^{n}g(X_i)$ and $Pg=\mathbb{E}g(X)$, the class $\mathcal{G}$ satisfies a uniform law of large numbers whenin probability; a strong ULLN uses almost-sure convergence.

Fix $\epsilon >0$ and choose finitely many brackets $[g_j^L,g_j^U]$ of $L^1(P)$ width at most $\epsilon$. The weak law of large numbers, simultaneously for their finitely many endpoints, givesIf $g_j^L\le g\le g_j^U$, bracketing both $P_{n}g$ and $Pg$ showsTaking the supremum gives a limit superior at most $\epsilon$. Since $\epsilon$ is arbitrary, the ULLN follows.

The Glivenko-Cantelli theorem states that for the empirical distribution function $F_n$ of iid real observations with distribution function $F$,almost surely.

Apply part (a) to $\mathcal{G}=\{1_{(-\infty ,t]}:t\in \mathbb{R}\}$. For each $\epsilon >0$, choose finitely many quantile cutpoints so that the $P$-mass between consecutive cutpoints is at most $\epsilon$, treating atoms as cutpoints themselves. Indicators at adjacent cutpoints give finite $L^1(P)$ brackets of width at most $\epsilon$. Using the strong law for the finite bracket endpoints and then intersecting the probability-one events for $\epsilon =1/m$ gives the almost-sure conclusion.

No. For each realized sample let $A_n=\{X_1,\dots ,X_n\}$. This is a finite Borel set, and continuity of $F$ makes $P(A_n)=0$ almost surely, while $P_n(A_n)=1$. Thereforealmost surely for every $n$, so the class of all Borel-set indicators cannot satisfy a ULLN.

A kernel for density estimation is an integrable function $K$ with $\int K=1$, usually also bounded and nonnegative, and its scaled version is $K_h(u)=h^{-1}K(u/h)$. For suitable $g_1,g_2$, their convolution is

Because the observations have length-biased density $g(u)=uf(u)/\mu$,Thus the exact bias is $(K_h\ast f)(x)-f(x)$, exactly the same as for the ordinary kernel density estimator based directly on observations from $f$.

Write $R(K)=\int K(u{)}^{2}du$. The estimator is the average of $Y_i(x)=\mu X_i^{-1}{K}_h(x-X_i)$, soIts mean is $K_h\ast f$. Integrating the pointwise variance therefore givesFinally, Cauchy-Schwarz inequality under density $f$ givesEquality would require $X$ to be constant almost surely, impossible for a density, so $\mu \bar{\mu }>1$. The ordinary KDE has the same negative term but leading integrated variance $R(K)/(nh)$; length-biased sampling strictly inflates it.

Let $r=\lceil \beta \rceil -1$. The Hölder class $\mathcal{H}(\beta ,L)$ consists of functions with $r$ continuous derivatives whose $r$th derivative satisfieswith the equivalent Lipschitz convention when $\beta$ is an integer.

The degree-$p$ local polynomial regression estimator minimizesand takes ${\hat{m}}_n(x;p,h)={\hat{b}}_0$. Put $X_i^T=Q((x_i-x)/h{)}^T$, $W_{ii}=K_h(x_i-x)$, and rescale $b_j$ by $h^j$. If $X^{T}WX$ is positive definite, weighted least squares givesDefineThen ${\hat{m}}_n=n^{-1}{\sum }_i{w}_i{Y}_i$. If $R$ has degree at most $p$, fitting the noiseless response $R(x_i)$ reproduces that polynomial exactly, and hence

For nonzero $a$, the polynomial $a^{T}Q(u)$ cannot vanish throughout $[0,1]$. Thereforeand compactness of the unit sphere makes its minimum eigenvalue ${\Lambda }_p$ positive. Choose $a(p)$ and $n_0(p)$ so that $nh\ge a(p)$ makes the supplied lower bound on ${\lambda }_0$ at least ${\Lambda }_p/4$. On the kernel support, $Q$ is bounded by a constant depending only on $p$, and $K_h\le 1/(2h)$. It follows that only $O(nh)$ weights are nonzero andPolynomial reproduction cancels the Taylor polynomial of degree $r\le p$. The Hölder remainder on $|x_i-x|\le h$ is at most $C_{p}L{h}^{\beta }$, so the squared bias is at most $C_p{L}^2{h}^{2\beta }$. Independence and $\mathrm{Var}{\epsilon }_i\le {\sigma }^2$ bound the variance by $C_p{\sigma }^2/(nh)$. Combining them uniformly in $x$ and $m$ proves

With densities $p,q$,By Cauchy-Schwarz inequality,Also $(\sqrt{p}-\sqrt{q}{)}^2\le |p-q|$, so $H^2\le 2\mathrm{TV}$.

The Hellinger affinity is $\rho (P,Q)=\int \sqrt{pq}d\mu =1-H^2/2$. Product densities and Fubini's theorem give $\rho (P^n,Q^n)=\rho (P,Q{)}^n$, hence

Le Cam two-point lemma states, for squared-error estimation at parameter points ${\theta }_0,{\theta }_1$, thatTake ${\theta }_0=0$, ${\theta }_1=\delta =1/(4n)$. The one-observation uniform densities overlap on length $1-\delta$, so $H^2(P_0,P_1)=2\delta$. ThereforeThe first distance inequality gives $\mathrm{TV}(P_0^n,P_1^n)\le 1/\sqrt{2}$. Le Cam's lemma now yieldsThis proves the claim with the displayed universal positive constant.