Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 223 / 2

Changing at most $n-h$ observations leaves at least $h$ original observations unchanged. At $\theta =0$, each unchanged observation has residual $y_i$ and loss at most $\rho (M_y)$. The sum of the $h$ smallest losses cannot exceed the sum over any chosen $h$ observations, soThe $L^1$ regularization term vanishes at zero.

By optimality and part a,Since the Euclidean norm is at most the $L^1$ norm,uniformly over all replacements of at most $n-h$ observations. The distance from the original finite estimate is therefore uniformly bounded, proving

For ${\theta }_{\gamma }=(\gamma ,0,\dots ,0{)}^T$, every moved observation has residualEvery original observation hasSelecting any $h$ losses and adding $\lambda \Vert {\theta }_{\gamma }{\Vert }_1=\lambda \gamma$ gives

Only $h-1$ original observations remain in $Z_{\gamma ,\tau }$, so the $h$ selected order statistics must include at least one moved observation. If $\Vert \theta {\Vert }_2\le \gamma -1$, then $|{\theta }_1|\le \gamma -1$, and its residual obeysMonotonicity of the symmetric loss, together with nonnegativity of the other loss and penalty terms, gives

Suppose for contradiction that $\Vert \hat{\theta }(Z_{\gamma ,\tau }){\Vert }_2\le M$ uniformly in $\gamma ,\tau$. Choose $\gamma >M+1$, so part d applies to every fitted value and gives $L(\hat{\theta })\ge \rho (\tau )$. For this fixed $\gamma$, part c gives the competing boundwhich is independent of $\tau$. Since $\rho (\tau )\to \infty$, choose $\tau$ so that the lower bound exceeds this upper bound, contradicting optimality. Thus $n-h+1$ replacements can make the estimate unbounded, andTogether with part b, the replacement breakdown point is exactly $(n-h)/n$.