Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 205 (Modern Statistical Methods)

With the usual objectivethe Karush-Kuhn-Tucker conditions for the Lasso areComparing the objective at ${\hat{\beta }}_{\lambda }$ and ${\beta }^0$, expanding the square, and cancelling the noise norm gives exactly

Choose $\lambda =A\sigma \sqrt{\log p/n}$. Since every column has norm $\sqrt{n}$, each $X_j^T\epsilon /n$ is sub-Gaussian with parameter $\sigma /\sqrt{n}$. A union bound givesOn the complementary event, Holder inequality and the triangle inequality bound the preceding right-hand side bywhich proves the stated prediction bound.

For the Dantzig selector, use the constraint $n^{-1}\Vert X^T(Y-X\beta ){\Vert }_{\infty }\le \lambda$. The Lasso KKT conditions make ${\hat{\beta }}_{\lambda }$ feasible, so a Dantzig minimizer $\tilde{\beta }$ has no larger ${\ell }^1$ norm. If $\tilde{\beta }$ were not the Lasso solution, uniqueness from invertibility of $X^{T}X$ would imply that some active coordinate $j$ has strict signed KKT slack. For small $\eta >0$, setOnly score coordinate $j$ changes, toward its feasible boundary, so ${\beta }_{\eta }$ remains feasible. For small $\eta$, signs on the active coordinates do not change, andby strict diagonal dominance. This contradicts Dantzig optimality, so the two estimators coincide.

Solved by gpt-5.6-sol high.

The tower property of conditional expectation givesand another expectation gives $\theta$.

Use the augmented inverse-probability-weighted estimatorCondition on the independently trained nuisance estimators. Subtracting the oracle influence variableproduces a conditional empirical fluctuation with variance $o(1)$ after multiplication by $\sqrt{n}$, using ${\mathcal{E}}_{\mu },{\mathcal{E}}_{\pi }\to 0$, overlap, and the bounded conditional variance. Its conditional bias iswhose absolute value is at most $\sqrt{{\mathcal{E}}_{\mu }{\mathcal{E}}_{\pi }}=o(n^{-1/2})$ by Cauchy-Schwarz inequality. ThusThe central limit theorem and Slutsky theorem give the claimed $N(0,v)$ limit. Without auxiliary data, use cross-fitting: split the sample into folds, train both nuisance estimators away from each observation's fold, and average the same score over held-out observations.

Solved by gpt-5.6-sol high.

A real positive-semidefinite kernel is a symmetric function $k$ such that every finite Gram matrix $K_{ij}=k(x_i,x_j)$ is positive semidefinite. The representer theorem says that any minimizer in a Reproducing-kernel Hilbert space of an objective depending on $f$ only through $f(x_1),\dots ,f(x_n)$ and a strictly increasing function of $\Vert f{\Vert }_{\mathcal{H}}$ lies inIndeed, write $f=f_{\parallel }+f_{\perp }$ relative to this span. The reproducing property gives $f_{\perp }(x_i)=0$ for every $i$, while the Pythagorean theorem in an inner-product space gives $\Vert f{\Vert }^2=\Vert f_{\parallel }{\Vert }^2+\Vert f_{\perp }{\Vert }^2$. Removing a nonzero perpendicular component preserves all data values and strictly decreases the penalty, proving the theorem.

Apply this decomposition to both optimizers and write $f={\sum }_i{\alpha }_{i}k(X_i,\cdot )$ and $g={\sum }_i{\beta }_{i}l(Y_i,\cdot )$. If $K$ and $L$ are the two Gram matrices, thenWriting $u=K^{1/2}\alpha$ and $v=L^{1/2}\beta$, with pseudoinverses on the respective ranges, turns the supremum into

Solved by gpt-5.6-sol high.

The graph has edges $1\to 2\leftarrow 3$, $1\to 4$, $2\to 4$, $3\to 5$, $4\to 5$, $4\to 6$, and $6\to 5$. Any D-separating set for 1 and 6 must contain 4 because of the directed path $1\to 4\to 6$. Conditioning on 4 activates the collider $1\to 4\leftarrow 2$ and, through its descendant, the collider $1\to 2\leftarrow 3$. The remaining route through $3\to 5\leftarrow 6$ is open exactly when 5 is conditioned on and 3 is not. Hence all separating sets, among the nonendpoint vertices, are

For the second graph, $2\perp \perp 5\mid 1,6$ forces colliders on the unblocked two-edge paths: $2\to 3\leftarrow 5$ and $2\to 4\leftarrow 5$. Acyclicity then forces $1\to 4$ and $6\to 4$. Thus its edges are

Solved by gpt-5.6-sol high.

Choose the given topological ordering. Since $j$ precedes $k$, $j$ is not a descendant of $k$; since they are nonadjacent, it is not a parent of $k$. The local Markov property of a directed acyclic graph says that a node is d-separated from all its nondescendants other than its parents by its parent set. Therefore $j$ and $k$ are d-separated by $\mathrm{pa}(k)$.

It follows that adjacency of $Z_1,Z_2$ is certified by rejecting every null hypothesisIf the vertices were nonadjacent, the theorem would supply one such separating parent set, whichever vertex comes later.

To certify that $X_1$ is a parent of $Y$, first rejectfor every $S\subseteq \{2,\dots ,d\}$, which forces adjacency. Then rejectfor every such $S$. If the adjacent edge were $Y\to X_1$, then $I\to X_1\leftarrow Y$ would orient $X_1$ as a collider, and the parent-set argument would provide a separator not containing $X_1$, contradicting the second collection of rejections. Hence the edge is $X_1\to Y$.

Solved by gpt-5.6-sol high.

The conditional multivariate normal distribution givesFor a jointly Gaussian vector, the residual is independent of the regressor, so $y=z^T\beta +\sigma e$ with $e\sim N(0,1)$ independent of $z$.

The Square-root Lasso minimizesAt a nonzero residual $R$, its KKT condition iswhich proves the required inequality.

Writing the response vector as $Y=f+\sigma e$, the reverse triangle inequality givesThe strong law of large numbers gives $\Vert e{\Vert }_2/\sqrt{n}{\to }_{P}1$, hence $\hat{\sigma }{\to }_P\sigma$.

Under $X\perp \perp Y\mid Z$, Gaussianity makes $e$ independent of $(X,Z)$ and hence of $R$. Conditionally on $R$, $R^{T}e/\Vert R{\Vert }_2\sim N(0,1)$. The remaining numerator term obeysCombining this with $\hat{\sigma }{\to }_P\sigma$ and Slutsky theorem proves the standard-normal limit.

Solved by gpt-5.6-sol high.