Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 218 (Statistical Learning in Practice)

Expanding the exponent of the Inverse Gaussian distribution givesThus the exponential dispersion family representationhasandThe standard cumulant identities yieldHence the variance function is $V(\mu )={\mu }^3$, and the canonical link function is $g(\mu )=\theta (\mu )=-1/(2{\mu }^2)$.

Solved by gpt-5.6-sol high.

Writing $d_i$ for difficulty, model1 assumes independent responseswith common dispersion. The estimates are ${\hat{\beta }}_0=9.0827$ and ${\hat{\beta }}_1=-1.4323$.

One extra difficulty level decreases the fitted inverse squared mean response time by $1.4323$. Since $\mu =({\beta }_0+{\beta }_{1}d{)}^{-1/2}$, this means that fitted mean response time increases with difficulty. The negative coefficient is therefore unsurprising; its sign looks counterintuitive only if the inverse-squared link function is ignored. The independence assumption is questionable because every subject contributes eight repeated responses.

Solved by gpt-5.6-sol high.

The Pearson residual iswhere the moment estimate of dispersion isIf the fitted model is adequate and the deviance residual sum is close to the Pearson statistic, then

Solved by gpt-5.6-sol high.

For response $j$ from subject $s$, model2 is the generalized linear mixed modelwith conditional independence given the random intercepts. The fitted values are ${\hat{\beta }}_0=8.9966$, ${\hat{\beta }}_1=-1.2317$, ${\hat{\tau }}^2=1.7533$, and fitted residual dispersion $0.7591$.

The random intercept models persistent between-subject differences and the resulting within-subject dependence among repeated measurements. That is the main feature absent from model1.

Solved by gpt-5.6-sol high.

Testwith a likelihood-ratio test. Model1 has three likelihood parameters and model2 has four, so their AIC values giveThe statistic isUsing a naive ${\chi }_1^2$ reference gives $p=0.0341$. Because the null variance lies on the boundary, the standard asymptotic reference is the mixture $\frac{1}{2}{\chi }_0^2+\frac{1}{2}{\chi }_1^2$, giving $p=0.0170$. Either calibration rejects at the five-percent level and supports a subject random effect.

Solved by gpt-5.6-sol high.

Order the observations so thatusing a fixed or randomized rule for ties. The L-nearest-neighbour classifier estimatesand predicts a class maximizing ${\hat{p}}_k(x)$.

Solved by gpt-5.6-sol high.

Conditional on $X_1,\dots ,X_n$, the $L$ selected class indicators are independent Bernoulli variables. ThereforeandTaking expectations proves the claim.

Solved by gpt-5.6-sol high.

Let $N_{\delta }$ count sample points in the intersection of $[0,1{]}^d$ with the ${\ell }^{\infty }$ ball of radius $\delta$ around $x$. That intersection has volume at least ${\delta }^d$, so $N_{\delta }\sim \mathrm{Bin}(n,q)$ with $q\ge M{\delta }^d$. The event $\Vert X_{(L)}-x{\Vert }_{\infty }>\delta$ implies $N_{\delta }<L$. Since $\mathrm{Var}(N_{\delta })\le nq$ and $nq\ge nM{\delta }^d\ge L$, Chebyshev inequality givesTaking the minimum with the trivial bound one proves the result.

Solved by gpt-5.6-sol high.

Put $R=\Vert X_{(L)}-x{\Vert }_{\infty }$ and ${\delta }_0=(2L/(nM){)}^{1/d}$. If ${\delta }_0\ge 1$, the claim follows from $R\le 1$. Otherwise, for $\delta \ge {\delta }_0$, part (c) givesThe tail-sum formula then yields

Solved by gpt-5.6-sol high.

The Lipschitz continuity of $p_1$ givesPart (d) therefore implies

Solved by gpt-5.6-sol high.

By the Cauchy-Schwarz inequality, part (b), and part (e),This bound tends uniformly to zero ifThe plug-in classifier excess-risk bound then shows that the misclassification risk of the nearest-neighbour classifier converges to the Bayes risk.

Solved by gpt-5.6-sol high.

The ${\xi }_i$ are slack variables of a support vector machine. The solid line is the support-vector-machine decision boundaryThe dashed lines are the two support-vector-machine margin boundariesEach is at perpendicular distance $1/\Vert \hat{\beta }{\Vert }_2$ from the decision boundary, so the full margin width is $2/\Vert \hat{\beta }{\Vert }_2$.

Solved by gpt-5.6-sol high.

For a monarch point, ${\hat{\xi }}_i=\max \{0,1-(\hat{\alpha }+X_i^{\top }\hat{\beta })\}$. Point P2 lies well beyond the monarch-side dashed margin, so a plausible value is $0$. P1 lies between the decision boundary and that margin, so a plausible value is about $0.4$. P3 lies on the wrong side of the decision boundary, so its slack exceeds one; about $1.3$ is plausible. Points P1 and P3 are support vectors, while P2 is not.

Solved by gpt-5.6-sol high.

Increasing $C$ raises the cost of slack variables of a support vector machine. The fit therefore generally accepts fewer margin violations and misclassifications, at the price of a larger $\Vert \hat{\beta }{\Vert }_2$ and hence a narrower support-vector-machine margin. Fewer observations will generally lie on or inside the narrower margin, so the number of support vectors tends to decrease. These are qualitative tendencies; individual counts need not vary monotonically for every data set.

Solved by gpt-5.6-sol high.

A training observation is misclassified only ifThe SVM constraint then forces ${\hat{\xi }}_i\ge 1$. HenceSumming and dividing by $n$ proves ${\widehat{\mathrm{Err}}}_{\mathrm{tr}}\le n^{-1}{\sum }_i{\hat{\xi }}_i$.

Solved by gpt-5.6-sol high.

The code computes Leave-one-out cross-validation. Ifthen observation $i$ is not a support vector. Removing it leaves the optimum unchanged, and the resulting classifier still classifies it correctly. A leave-one-out error can therefore occur only for an observation on or inside the margin, which provesThus the fraction of training observations on or inside the margin is an upper bound on leave-one-out error. One can refit only after deleting support vectors, reusing the full fit for every other observation. Alternatively, K-fold cross-validation needs only $K$ fits and is often preferable for larger data sets.

Solved by gpt-5.6-sol high.

A real positive-semidefinite kernel is a symmetric function $k:X\times X\to \mathbb{R}$ such that every finite Gram matrix $(k(x_i,x_j))$ is positive semidefinite. The Moore-Aronszajn theorem says that there are a Hilbert space $H$ and a feature map $\varphi :X\to H$ such thatEquivalently, $H$ can be chosen as the unique Reproducing-kernel Hilbert space with reproducing kernel $k$.

Solved by gpt-5.6-sol high.

Let $(e_{r,a},e_{r,b}{)}_{r=1}^L$ be the standard orthonormal basis of ${\mathbb{R}}^{2L}$ and defineThenThus $g$ is a positive-semidefinite kernel.

Solved by gpt-5.6-sol high.

The empirical kernel covariance operator $\Sigma$ is self-adjoint and positive semidefinite. Maximize $⟨u,\Sigma u⟩$ subject to $⟨u,u⟩=1$. The first variation of the Lagrange multiplier functional givesso $\Sigma v=\lambda v$. Taking the inner product with $v$ gives

Solved by gpt-5.6-sol high.

Positive definiteness of $K$ makes the vectors $\varphi (x_1),\dots ,\varphi (x_n)$ linearly independent. In particular $\Sigma \ne 0$, so the maximum Rayleigh quotient is positive and $\lambda >0$. The eigenvector equation giveswhich lies in their span. Hence $v={\sum }_i{\alpha }_i\varphi (x_i)$ for some $\alpha \in {\mathbb{R}}^n$.

Solved by gpt-5.6-sol high.

Substituting $v={\sum }_j{\alpha }_j\varphi (x_j)$ into $\Sigma v=\lambda v$ givesLinear independence yields $K\alpha =n\lambda \alpha$. The unit-norm constraint gives

Solved by gpt-5.6-sol high.

The feature space may be extremely high-dimensional or infinite-dimensional, and $\varphi$ may be known only implicitly. Instead, compute the leading eigenvector $\alpha$ of the kernel matrix $K$, normalize it by ${\alpha }^{\top }K\alpha =1$, and use the kernel trick:This requires only kernel evaluations.

Solved by gpt-5.6-sol high.

Kernel principal component analysis can represent nonlinear low-dimensional structure by performing linear PCA in a nonlinear feature space. It can also work directly with structured objects such as strings through a kernel, without assigning them explicit finite-dimensional coordinates. Both capabilities are unavailable to ordinary linear PCA on the original variables.

Solved by gpt-5.6-sol high.

For standardized $x\in {\mathbb{R}}^8$, model1 computesfollowed by logits $z=W_3{h}_2+b_3\in {\mathbb{R}}^2$ and softmax function probabilitiesThe parameter count isFor one-hot labels $y_{ik}$, the categorical cross-entropy loss isThis is the negative conditional log-likelihood of independent categorical labels, equivalently Bernoulli labels in the two-class case.

Solved by gpt-5.6-sol high.

Stochastic gradient descent replaces the full empirical-loss gradient by the gradient on a randomly ordered observation or mini-batch, then updates $\theta \leftarrow \theta -\eta \hat{\nabla }L(\theta )$. The training half contains $768/2=384$ observations, so batches of 16 give $384/16=24$ updates per epoch. Over 100 epochs every parameter is updated $2400$ times.

Solved by gpt-5.6-sol high.

The neural-network likelihood is nonconvex, so optimization can stop at a local optimum or saddle rather than a global maximum. One hundred epochs may be insufficient for convergence. Mini-batch gradient noise together with a fixed positive learning rate can keep the iterates fluctuating around a stationary point rather than reaching it exactly. Any of these prevents the final parameters from being exact maximum-likelihood estimators.

Solved by gpt-5.6-sol high.

The dashed curve is training accuracy: it continues to rise as optimization adapts to the training observations. The solid curve is testing accuracy: it peaks near 10 epochs and then declines. The widening gap is overfitting.

A sensible choice is about 10 epochs, selected by early stopping at the maximum validation accuracy. In a proper analysis, a validation set rather than the final test set should choose this epoch. Early stopping is an implicit regularization method because it limits how far the parameters can adapt to training-specific noise.

Solved by gpt-5.6-sol high.