Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 223 (Robust Statistics)

Put $U=X/\sigma$. The standardized efficient score function for scale isAn optimal B-robust scale M-estimator solveswhere the optimal bounded score has the clipped-score formThe constants $A,z$ enforce Fisher consistency, the chosen normalization, and the clipping bound $b$ on the influence function.

For the standard normal distribution, $f^{\prime }(u)/f(u)=-u$, hence $\Lambda (u)=u^2-1$. The score therefore simplifies towith $z$ chosen so that $\mathbb{E}\psi (Z)=0$ for $Z\sim N(0,1)$.

Write $X=\sigma Z$ with $Z\sim F_1$. Thenwhere the distribution of $W=\log Z^2$ does not depend on $\sigma$. Consequentlywhich is a location family. Since $\sigma >0$, the transformed parameter $\theta =\log {\sigma }^2$ ranges over all of $\mathbb{R}$; the paper's restriction $\theta >0$ is unnecessary.

Let $W=Y-\theta \sim G_0$ and suppose $a={\mathbb{E}}_{G_0}[{\psi }^{\prime }(W)]\ne 0$. A first-order expansion of the estimating equation $n^{-1}{\sum }_i\psi (Y_i-T_n)=0$ gives the asymptotic linear representationThe central limit theorem therefore yields

The transformation $g(t)=e^{t/2}$ satisfies $g(\theta )=\sigma$ and $g^{\prime }(\theta )=\sigma /2$. Applying the delta method to part c givesThus exponentiating half the robust location estimate produces an asymptotically normal scale estimator.

Changing at most $n-h$ observations leaves at least $h$ original observations unchanged. At $\theta =0$, each unchanged observation has residual $y_i$ and loss at most $\rho (M_y)$. The sum of the $h$ smallest losses cannot exceed the sum over any chosen $h$ observations, soThe $L^1$ regularization term vanishes at zero.

By optimality and part a,Since the Euclidean norm is at most the $L^1$ norm,uniformly over all replacements of at most $n-h$ observations. The distance from the original finite estimate is therefore uniformly bounded, proving

For ${\theta }_{\gamma }=(\gamma ,0,\dots ,0{)}^T$, every moved observation has residualEvery original observation hasSelecting any $h$ losses and adding $\lambda \Vert {\theta }_{\gamma }{\Vert }_1=\lambda \gamma$ gives

Only $h-1$ original observations remain in $Z_{\gamma ,\tau }$, so the $h$ selected order statistics must include at least one moved observation. If $\Vert \theta {\Vert }_2\le \gamma -1$, then $|{\theta }_1|\le \gamma -1$, and its residual obeysMonotonicity of the symmetric loss, together with nonnegativity of the other loss and penalty terms, gives

Suppose for contradiction that $\Vert \hat{\theta }(Z_{\gamma ,\tau }){\Vert }_2\le M$ uniformly in $\gamma ,\tau$. Choose $\gamma >M+1$, so part d applies to every fitted value and gives $L(\hat{\theta })\ge \rho (\tau )$. For this fixed $\gamma$, part c gives the competing boundwhich is independent of $\tau$. Since $\rho (\tau )\to \infty$, choose $\tau$ so that the lower bound exceeds this upper bound, contradicting optimality. Thus $n-h+1$ replacements can make the estimate unbounded, andTogether with part b, the replacement breakdown point is exactly $(n-h)/n$.

For a location M-functional defined byits influence function isprovided the denominator is nonzero. A bounded score therefore gives a bounded influence function.

The Huber location estimator usesaway from the two corners. At $F=N(\theta ,1)$, symmetry gives $T(F)=\theta$, and for $Z\sim N(0,1)$,Hencewhich is bounded in $x$.

For a randomized test $\phi (X_1,\dots ,X_n)\in [0,1]$, level $\alpha$ over the null neighborhood meansA maximin test maximizes its worst-case poweramong all tests satisfying that robust level constraint.

For $P_0=N(-{\theta }_0,1)$ and $P_1=N({\theta }_0,1)$, the nominal log-likelihood ratio for one observation is $2{\theta }_{0}x$. Under sufficiently small epsilon-contamination neighborhoods, the least-favourable pair clips this likelihood ratio between two constants. Symmetry turns its log into a positive multiple of the winsorized scoreThe robust form of the Neyman-Pearson lemma therefore rejects for large values ofwith boundary randomization and threshold chosen so that the worst-case null rejection probability is $\alpha$. Extreme observations contribute only $\pm c$, preventing a few contaminants from dominating the test.

The test statistic is generated by the functional $T(F)=\int q_c(x)dF(x)$. At $F=P_0$, its influence function isBecause $|q_c(x)|\le c$, this influence function is bounded. Multiplying $q_c$ by the constant inherited from the clipped log-likelihood ratio only rescales the same bounded expression.

After the replacement, the objective isby Euclidean norm duality. It is nonnegative and equals zero exactly at $\mu ={\mu }_0$, so ${\mu }_0$ is the unique minimizer.

DefineFor every unit-ball vector $v$, the triangle inequality givesTaking the supremum and using optimality, $R_X(\hat{\mu })\le R_X({\mu }_0)$, yields

Choose a fixed $\alpha <1/2$. On the stated high-probability event, strictly more than half of the block means satisfy, simultaneously for every $\Vert v{\Vert }_2\le 1$,The median of a collection with a strict majority in an interval lies in that interval. Hence the same bound holds for $|{\mathrm{MOM}}_k(Xv)-{\mu }_0^{T}v|$ uniformly in $v$. Part b then giveswith probability at least $1-e^{-k/c_{\alpha }}$. Renaming the constant proves the claim.

Use the quoted event with, for example, $\alpha =1/4$, so at least $3k/4$ clean block means obey the uniform bound. Replacing at most $ϵn$ observations can corrupt at most $ϵn$ blocks. If $k\ge 4\max \{nϵ,1\}$, at least $3k/4-ϵn\ge k/2$ block means remain both uncorrupted and good; adjusting constants handles equality and integer rounding. Their strict majority forces every projected median to obey the same uniform bound. Applying part b and absorbing fixed constants giveswith probability at least $1-e^{-c^{\prime }k}$.

For $\Sigma =I_d$, $\mathrm{tr}(\Sigma )=d$ and $\Vert \Sigma {\Vert }_2=1$. Taking $k$ of order $\max \{nϵ,1\}$ in part d givesFor Gaussian data under Huber contamination, the Tukey median has the sharper dependenceup to universal constants in its valid contamination range. Both have the same sampling term $\sqrt{d/n}$, but the Tukey median is linear rather than square-root in the contamination fraction.