Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 205 (Modern Statistical Methods)

For a convex function $f:{\mathbb{R}}^d\to \mathbb{R}$, its subdifferential at $x$ isIts elements are the subgradients of $f$ at $x$.

Only the $k$th block can have nonzero subgradient coordinates. If ${\beta }^{(k)}\ne 0$, the Cauchy-Schwarz inequality shows that the unique supporting vector is ${\beta }^{(k)}/\Vert {\beta }^{(k)}{\Vert }_2$. At zero, the defining inequality is $\Vert h{\Vert }_2\ge u^{T}h$ for every block vector $h$, which is equivalent to $\Vert u{\Vert }_2\le 1$. Thus

The differentiable loss has gradient $X^T(X\beta -Y)/n$. The subdifferential sum rule and part (b) givewhere blockwise

Suppose ${\hat{\beta }}_1$ and ${\hat{\beta }}_2$ minimize $Q$. Their midpoint is also a minimizer because $Q$ is a convex function. The group penalty is convex, while the squared Euclidean norm is strictly convex in the fitted value. If $X{\hat{\beta }}_1\ne X{\hat{\beta }}_2$, strict convexity would make the midpoint objective strictly smaller than the minimum. Hence $X{\hat{\beta }}_1=X{\hat{\beta }}_2$, so the fitted values are unique.

Part (d) makes the residual and therefore $\hat{\nu }=X^T(Y-X\hat{\beta })$ unique, so $E$ is unique. The Karush-Kuhn-Tucker conditions from part (c) imply that every nonzero block satisfies $\Vert {\hat{\nu }}^{(k)}{\Vert }_2=n\lambda \sqrt{m}$. Hence ${\hat{\beta }}^{(k)}=0$ for $k\notin E$.

Any two minimizers have the same fitted value and vanish outside $E$. Their difference $h$ is therefore supported on $E$ and satisfies $\tilde{X}{h}_E=0$. If $\tilde{X}$ has full column rank, then $h_E=0$, proving uniqueness of $\hat{\beta }$.

Optimality gives $Q(\hat{\beta })\le Q({\beta }^0)$. Substitute $Y=X{\beta }^0+\epsilon$, expand both squared norms, and cancel $\Vert \epsilon {\Vert }_2^2/(2n)$. Rearranging givesThis exposes a factor-of-two typo in the paper: its requested display has $1/n$ rather than $1/(2n)$ on the left while leaving the right side unchanged. For the objective printed in the paper, the displayed inequality above is the correct basic inequality. Part (b) explicitly asks us to use the stronger stated version, so the subsequent argument proceeds from that requested premise.

Let $\delta =\hat{\beta }-{\beta }^0$. Standard sub-Gaussian random variable and Gaussian-product concentration gives, for $n>\log p$,Thus $A=4$ makes this probability at most $2/p$. On the complementary event, the basic inequality and Holder inequality giveThe parenthesis is at most $2\Vert \delta {\Vert }_1$ by the triangle inequality and at most $2\Vert {\beta }^0{\Vert }_1$ by $\Vert \delta {\Vert }_1\le \Vert \hat{\beta }{\Vert }_1+\Vert {\beta }^0{\Vert }_1$. Substituting $\lambda =A\sigma v\sqrt{\log p/n}$ proves ${\Omega }_1$, whose probability tends to one.

Put $s=\Vert \hat{\beta }{\Vert }_0+\Vert {\beta }^0{\Vert }_0$. Since $\delta$ has at most $s$ nonzero coordinates, $\Vert \delta {\Vert }_1^2\le s\Vert \delta {\Vert }_2^2$. On ${\Omega }_2$,where $\hat{\Sigma }=X^{T}X/n$. Write $R={\delta }^T\hat{\Sigma }\delta$. On ${\Omega }_1$,Squaring after division by $\sqrt{R}$ proves

A symmetric function $k:{\mathcal{X}}^2\to \mathbb{R}$ is a positive-definite kernel when ${\sum }_{i,j}{a}_i{a}_{j}k(x_i,x_j)\ge 0$ for every finite set of points and real coefficients. A Reproducing-kernel Hilbert space $\mathcal{H}$ is a Hilbert space of functions whose evaluation maps are continuous. Its reproducing kernel satisfies $k(x,\cdot )\in \mathcal{H}$ and $f(x)=⟨f,k(x,\cdot ){⟩}_{\mathcal{H}}$.

Let $U$ be uniform on $[0,2\pi ]$, and independently let $W$ have densityThe supplied Fourier transform identity, after the change of Fourier convention, givesAlso ${\mathbb{E}}_U[\cos (a+U)\cos (b+U)]=\frac{1}{2}\cos (a-b)$. Thereforeso one may take $c=1$.

For any $x_1,\dots ,x_N$ and $a_1,\dots ,a_N$, part (b) and linearity of expectation giveSymmetry is immediate, so $k$ is a positive-definite kernel.

On a grid $G\subset [-L,L]$ with mesh proportional to $\epsilon$, Hoeffding inequality and a union bound giveThe density of $W$ has exponential tails. A Chernoff bound therefore shows that ${\ell }^{-1}{\sum }_i|W_i|$ is bounded by an absolute constant except on an event of probability $e^{-c^{\prime }\ell }$. On that event, both the empirical kernel and $k$ are uniformly Lipschitz, so every pair $(x,y)$ is approximated by its nearest grid pair with total error at most $\epsilon /2$. Enlarging constants and using $0<\epsilon \le 1$ yields

A p-value $p_i$ for $H_i$ is super-uniform under that null: ${\mathbb{P}}_{H_i}(p_i\le u)\le u$ for every $u\in [0,1]$. For the Benjamini-Hochberg procedure, order $p_{(1)}\le \cdots \le p_{(m)}$, setwith $\hat{k}=0$ if the set is empty, and reject the hypotheses whose p-values are at most $p_{(\hat{k})}$.

Let $R$ be the number of rejections and $I_0$ the true-null indices. For $i\in I_0$, remove $p_i$ and let $R_i$ be the number determined by the corresponding leave-one-out step-up rule. On $\{i\text{ rejected},R=r\}$ one has $p_i\le \alpha r/m$ and $R_i=r$, while $R_i$ is independent of $p_i$. Hence super-uniformity givesSumming over $i\in I_0$ proves that the false discovery rate is at most $|I_0|\alpha /m\le \alpha$.

Under the intersection null, every rejection is false, so the false-discovery proportion is $1_{\{R>0\}}$ and its expectation is the familywise error rate. Because the p-values are independent and exactly uniform, every super-uniform inequality in part (b) is an equality. Here $|I_0|=m$, and thereforeEquivalently, the order-statistic identity in the hint gives the same equality by induction on $m$.

The ridge regression estimator minimizesIts gradient vanishes exactly when $(X^{T}X+\lambda I)\beta =X^{T}Y$. Since $\lambda >0$ makes this matrix positive definite,

Use a singular value decomposition $X=UD{V}^T$. ThenEach nonzero singular value $d$ contributes $d/(d^2+\lambda )\to d^{-1}$, while each zero one contributes zero. Thususing the Moore-Penrose inverse.

Put $Z=[X,W]$. Its ridgeless minimum-norm fit is $Z^T(Z{Z}^T{)}^{+}Y$, so its first $p$ coordinates areThe strong law of large numbers applied entrywise gives $W{W}^T\to \lambda I_n$ almost surely as $d\to \infty$. Continuity of inversion then yieldsalmost surely, where the middle equality is the push-through identity.

Because $x_{\ast }\sim N_p(0,I)$ is independent, conditional prediction risk equals $\mathbb{E}(\Vert {\hat{\beta }}_{\lambda }-{\beta }^0{\Vert }_2^2\mid X)$. With $\hat{\Sigma }=X^{T}X/n$ and $\lambda =n\ell$,The noise term has conditional mean zero and covariance $\frac{{\sigma }^2}{n}(\hat{\Sigma }+\ell I{)}^{-1}\hat{\Sigma }(\hat{\Sigma }+\ell I{)}^{-1}$. Taking squared norms proves

Apply the stated deterministic equivalent for the squared resolvent with ${\Theta }_p={\beta }^0({\beta }^0{)}^T/r^2$. The bias term tends to $-{\ell }^2{r}^2{m}^{\prime }(\ell )$. For the variance useNormalized traces and $p/n\to \gamma$ therefore giveThus the conclusion printed in the paper also has both signs involving $m^{\prime }$ reversed. The hypotheses as printed imply the formula above; they cannot imply the requested one because $(\hat{\Sigma }+\ell I{)}^{-2}$ is positive definite while $m^{\prime }(\ell )\le 0$ for a resolvent limit.