Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 218 (Statistical Learning in Practice)

The first fit is a Poisson regression with independent responsesThe fitted intercept gives $e^{3.8879}$ expected visitors when spending on both services is zero. Holding advert2 fixed, increasing advert1 by one unit, namely one hundred pounds, multiplies the expected visitor count by $e^{0.1923}\simeq 1.212$.

The second fit is a Quasi-Poisson regression. It retains the logarithmic link function and mean modelbut assumes only $\mathrm{Var}(Y_i)=\varphi {\mu }_i$ for an unknown dispersion parameter $\varphi$, rather than a complete Poisson distribution.

The quasi-Poisson standard errors are the Poisson standard errors multiplied by $\sqrt{\hat{\varphi }}$, and the Pearson estimator divides the Pearson chi-squared statistic by the residual degrees of freedom $40-3=37$. Therefore the requested sum of squared Pearson residuals isup to the rounding in the printed standard errors.

The third fit is a negative binomial regression. The Poisson model is obtained at the boundary where the negative-binomial overdispersion tends to zero, so the likelihood-ratio test has the asymptotic null distribution $\frac{1}{2}{\chi }_0^2+\frac{1}{2}{\chi }_1^2$. Since model3 has one additional parameter,The supplied output gives $\mathbb{P}({\chi }_1^2\ge D)=0.04608555$, and hence the boundary-corrected p-value is

The stated mean and variance identify ${\lambda }_i$ as a gamma distribution with shape ${\nu }_i$ and scale ${\alpha }_i$. A Poisson-gamma mixture is negative binomial, withWhen ${\nu }_i=\nu$, this is exactly the constant-shape variance model fitted by the negative binomial regression, so the most appropriate printed p-value is $0.0798$ from model3.

When ${\alpha }_i=\alpha$, the law of total variance givesThis is the quasi-Poisson mean-variance relation with constant dispersion, so the most appropriate printed p-value is $0.0982$ from model2.

Among all depth-one regression tree splits, the best split separates the second observation from the first and third by cutting advert1 between $0.89$ and $1.13$. The test point with advert1 equal to zero reaches the leaf containing responses $60$ and $53$, so the output is their arithmetic mean,This decision stump makes a piecewise-constant prediction far outside the observed predictor range and cannot extrapolate the spending trend towards the origin. Its shallow structure and leaf averaging keep its variance of an estimator modest, while that extrapolation failure can produce substantial bias of an estimator.

The ridge regression estimator is the elastic net at $\alpha =0$, while the lasso regression estimator is the elastic net at $\alpha =1$.

Put $z={\hat{\beta }}_{\mathrm{OLS}}=X^{T}Y/n$. Under $X^{T}X=n{I}_p$, the objective separates by coordinates. Completing the square and applying the soft-thresholding operator $S(u,t)=\mathrm{sign}(u)(|u|-t{)}_{+}$ givesFor a fixed $z_j$, its magnitude lies between the ridge endpoint $|z_j|/(1+\lambda )$ and the lasso endpoint $(|z_j|-\lambda {)}_{+}$; this follows directly on the two intervals $\lambda \alpha \ge |z_j|$ and $\lambda \alpha <|z_j|$ by cross-multiplication. Thus the stated endpoint inequality holds.

For $alpha>0$ and $z_j\ne 0$, the coordinate first vanishes when the soft threshold reaches $|z_j|$, soThis is strictly decreasing in $\alpha$, and it diverges to infinity as $\alpha \downarrow 0$. This agrees with the fact that pure ridge shrinkage does not set a nonzero coordinate exactly to zero at any finite penalty.

Writing the horizontal coordinate as $u=\log \lambda$, so that $\lambda =e^u$, the two coefficient paths at $\alpha =1/2$ areEquivalently, as functions of $\lambda$, replace every $e^u$ by $\lambda$. The paths reach zero at $lambda=2.82$ and $lambda=0.82$, respectively.

Let $z={\hat{\beta }}_{\mathrm{OLS}}$ and $G=X^{T}X/n=\left(\begin{array}{cc}1& \rho \\ \rho & 1\end{array}\right)$. While both lasso coordinates are positive, the Karush-Kuhn-Tucker conditions giveHenceAssume without loss of generality that $z_1\le z_2$. After the first coordinate vanishes, the second remains active until $lambda=z_2+\rho z_1$. The zero vector satisfies the KKT conditions exactly when $lambda\ge \Vert Gz{\Vert }_{\infty }$; for $z_1\le z_2$ and $-1<\rho <1$, this norm is $z_2+\rho z_1$. Thereforeandwhich is independent of $\rho$.

The conditional class probability is $p_k(x)=\mathbb{P}(Y=k\mid X=x)$. A Bayes classifier choosesand its Bayes risk isA sequence of classifiers is consistent when $R(h_n)\to R_{\mathrm{Bayes}}$ as $n\to \infty$, with convergence interpreted in probability or in expectation according to whether the training sample is conditioned upon.

The $L$-nearest-neighbour classifier finds the $L$ training predictors nearest to $x$ and returns the majority class among their labels. Increasing $L$ averages more labels and reduces variance of an estimator, but uses observations farther from $x$ and therefore increases bias of an estimator; decreasing $L$ reverses this bias-variance tradeoff.

For two classes, write $m(x)=\min \{p_1(x),p_2(x)\}$, the conditional Bayes error. The nearest-neighbour label and the test label become conditionally independent draws from the same local class distribution, so the limiting conditional error of one-nearest-neighbour classification isThe assumption gives $c\le m(x)\le 1/2-c$. Its excess over the conditional Bayes error isAfter taking expectations, the limiting risk remains at least $2c^2$ above the Bayes risk, so one-nearest-neighbour classification is not consistent.

The limiting conditional error of one-nearest-neighbour classification is $1-{\sum }_k{p}_k(x{)}^2$. Put $M={\max }_k{p}_k(x)$ and $r=1-M$. By the Cauchy-Schwarz inequality, the other $K-1$ probabilities satisfyConsequentlyNow $\mathbb{E}r=R_{\mathrm{Bayes}}$, and Jensen inequality gives $\mathbb{E}{r}^2\ge (\mathbb{E}r{)}^2$. Taking expectations proves

Encode the two classes by the standard basis vectors of ${\mathbb{R}}^2$. With rectified linear unit $r(t)=\max (t,0)$ applied coordinatewise and softmax function $s_j(z)=e^{z_j}/{\sum }_k{e}^{z_k}$, the fitted feedforward neural network iswhere $W_1\in {\mathbb{R}}^{40\times 40}$, $b_1\in {\mathbb{R}}^{40}$, $W_2\in {\mathbb{R}}^{20\times 40}$, $b_2\in {\mathbb{R}}^{20}$, $W_3\in {\mathbb{R}}^{2\times 20}$, and $b_3\in {\mathbb{R}}^2$. The number of trainable parameters is

Training minimizes the empirical categorical cross-entropy lossby stochastic gradient descent. The independent validation set monitors generalization, while the small fixed number of epochs limits how long the network can fit training noise; using validation loss for early stopping would make this safeguard explicit. A forward pass computes all layer activations, class probabilities, and the mini-batch loss. There are $10000/2=5000$ training mini-batches per epoch and hence $5\cdot 5000=25000$ training forward passes, in addition to validation evaluation after each epoch.

The usual Akaike information criterion correction assumes a regular maximum-likelihood fit with a meaningful fixed parameter dimension. Here stochastic optimization stopped after five epochs need not attain the maximum likelihood estimator, and neural-network symmetries, inactive units, and heavy overparameterization make the raw count $2502$ a poor effective dimension. Either failure invalidates a direct AIC comparison with an ordinary logistic regression.

The proposed function $g(\eta )=\eta \Phi (\eta )$ is the Gaussian error linear unit. Unlike ReLU, it is smooth at zero and has a nonzero gradient on much of the negative half-line, reducing dead hidden units and making gradient optimization smoother. It is less computationally convenient because evaluating the standard normal cumulative distribution function is costlier than taking a maximum, and it does not produce ReLU's exact sparse zero activations.

The intended quantity is the Leave-one-out cross-validation errorwhere ${\hat{h}}^{(-i)}$ is trained without observation $i$ and the loss is the zero-one misclassification loss. Computing it literally requires fitting $10000$ neural networks, which is prohibitively expensive.

The assignment nn.model.i <- nn.model does not construct a fresh untrained Keras model: it aliases an object whose weights were already fitted using every training observation, including the nominally held-out one, and repeated fits continue mutating those weights. This data leakage makes metric2 severely optimistic. Moreover, random leave-one-out validation among reviews from 2012--2025 does not reproduce the dataset shift to new recent reviews that metric1 measures.

Model1 is the simple linear regressionModel2 is a random-intercept linear mixed modelwith the random intercepts and errors mutually independent.

Repeated measurements from one person can share persistent unobserved spending tendencies, violating model1's independent-error assumption and shifting that person's baseline. Model2 represents this clustered data through a common random intercept $b_i$, which induces within-person covariance $\mathrm{Cov}(Y_{ij},Y_{ik}\mid X)={\sigma }^2$ for $j\ne k$.

The residual deviance of a Gaussian GLM is its residual sum of squares, $154.30$. Thus the empirical training error for squared-error loss isIf training error is defined as the unnormalized total loss, its value is $154.30$.

The first column of the model matrix is the all-ones intercept column. Hence $A_{11}={\sum }_{i,j}1=100$.

The second cross-product entry is $A_{12}={\sum }_{i,j}{X}_{ij}$. Regressing earned on the ten person indicators without an intercept makes $(g_x{)}_i$ the mean of that person's ten earned values. Therefore

Let $q=(\alpha ,\beta {)}^T$, letand recover the sufficient cross-products from the model1 normal equations by setting $c=A{\hat{q}}_0$ and $s=154.30+{\hat{q}}_0^{T}A{\hat{q}}_0$. Thenis the total squared residual about the fixed line. The squared sum of the ten residuals within each person, summed across people, isFor one person's ten observations, the marginal covariance matrix is ${\tau }^2{I}_{10}+{\sigma }^{2}11^T$. The matrix determinant lemma and Sherman–Morrison formula therefore give, up to an additive constant, twice the negative marginal log-likelihoodBecause the model was fitted with REML = FALSE, it minimizes this ordinary marginal maximum-likelihood objective. Thus $(V1,V2,V3,V4)$ belongs to the stated argmin over $alpha,\beta \in \mathbb{R}$ and $sigma,\tau >0$.