Past exam of the mathematics course of the University of Cambridge / 2002 / iii / Paper 36 / 2

For logistic regression, set ${\eta }_i=x_i^{\mathsf{T}}\beta$ and $p_i=(1+e^{-{\eta }_i}{)}^{-1}$. Its Bernoulli log-likelihood, score function and Fisher information arewhere $W=\mathrm{diag}\{p_i(1-p_i)\}$. Solve $U(\beta )=0$ by Fisher scoring or iteratively reweighted least squares. One scoring step isEquivalently regress the working response $z_i={\eta }_i+(y_i-p_i)/[p_i(1-p_i)]$ on $X$ using weights $p_i(1-p_i)$. Iterate until the likelihood and coefficients stabilize, with rank and convergence checks. Separation in logistic regression can prevent a finite maximum: if a predictor direction strictly separates the zeros and ones, moving indefinitely along it improves the likelihood rather than producing a finite root.

Under the unrestricted model, each individual probability is estimated independently as ${\hat{p}}_i=y_i$. Every observed binary outcome then has probability one, so, using boundary limits,The resulting residual deviance is $D=-2\ell (\hat{\beta })$. It is a legitimate likelihood contrast and can be used to compare fitted models. However, individual Bernoulli deviance need not have a chi-squared calibration: the saturated model has one boundary-valued parameter for each single observation, and its dimension grows with $n$. The fixed-dimensional regularity behind Wilks theorem does not justify a ${\chi }_{n-p}^2$ reference here.

For a concrete demonstration, let the true model contain only an intercept and have success probability $1/2$. Its fitted probability is $\bar{y}$, andwhereas a ${\chi }_{n-1}^2$ variable divided by $n$ converges to one. Thus treating the raw Bernoulli residual deviance as an ordinary absolute goodness-of-fit statistic can reject a correct model systematically. In contrast, deviance differences between fixed-dimensional nested logistic models have their usual asymptotic chi-squared reference under the relevant regularity assumptions. For absolute fit use meaningful replicated covariate groups and grouped-binomial logistic regression, or assess residual patterns and prediction on held-out data; a model-specific parametric bootstrap can calibrate a chosen statistic.

The family-history table contains $462$ men, of whom $160$ have the response present, giving an overall observed fraction $160/462=0.3463$. The unadjusted fractions are $96/192=0.5$ with a positive family history and $64/270=0.2370$ without one. Their crude odds ratio is $(96/96)/(64/206)=3.219$; this is not the covariate-adjusted effect from the fitted logistic regression.

The full model has an intercept and nine slopes, hence $462-10=452$ residual degrees of freedom. The binomial distribution dispersion is fixed at one. Although the software labels coefficient-to-standard-error ratios as t values, here they are asymptotic normal Wald statistics, not statistics with an estimated Gaussian residual variance and an exact Student reference. The full conditional coefficients provide evidence for tobacco, cholesterol, family history, behaviour score and age: their absolute Wald ratios are approximately $2.99,2.92,4.06,3.22,3.73$. The other four ratios are smaller in magnitude, especially alcohol at about $0.027$. These are conditional associations; a weak adjusted coefficient does not by itself establish absence of an underlying association, and multicollinearity can affect the estimates.

The null deviance is $596.1084$ on $461$ degrees of freedom and the full residual deviance is $472.14$ on $452$. Their difference $123.9684$ has a nominal nine-degree-of-freedom likelihood-ratio test reference for the nine slopes jointly. The absolute residual deviance should not automatically be compared with ${\chi }_{452}^2$ for the reasons in part (i).

The shown stepwise selection by the Akaike information criterion follows a backward path. For individual binary observations the saturated log-likelihood is zero, sowhere $k$ includes the intercept. Removing a one-parameter term improves AIC exactly when the increased deviance is less than two. The successive deletions are alcohol, adiposity, blood pressure and obesity. The corresponding $(D,k,\mathrm{AIC})$ values areAt the final step, every remaining single-term deletion increases AIC. This is a local stopping condition of the search, not a proof that it found the best subset among all possible models. Obesity is retained for several steps because its successive deletion costs differ after the other terms are removed; the procedure does not simply sort the initial Wald ratios.

For the selected fit, write $H=1$ for positive family history and $H=0$ otherwise. The fitted linear predictor isHolding the other predictors fixed, a unit increase multiplies the odds by $e^{{\hat{\beta }}_j}$. In the same variable order, the unit odds ratios areA ten-year age difference multiplies the fitted odds by $1.656$, and ten behaviour-score points by $1.449$. Family history has adjusted odds ratio about $2.48$, with ordinary fitted-model $95\%$ confidence interval $\exp (0.908171\pm 1.96\times 0.225603)\simeq [1.59,3.86]$. The intercept is the log odds at zero numerical covariates and absent family history; that extrapolated baseline is not a typical man's risk. Odds ratios are not risk ratios.

The selected model's residual degrees of freedom are $456$, consistent with its six coefficients. All five retained slopes have nominal normal-Wald $p$ values below $0.004$ in that model, but these tests and confidence intervals do not incorporate the prior model selection. Its improvement over the intercept-only fit is $596.1084-475.6856=120.4228$ on five nominal degrees of freedom; it loses only $3.5456$ deviance units relative to the nine-slope fit while using four fewer parameters. Selection optimism remains relevant to predictive performance. Check possible nonlinear continuous-predictor effects, influential observations and clinically motivated interactions; assess out-of-sample discrimination and agreement of predicted probabilities with observed frequencies by cross-validation or external data. The printed output alone does not supply those diagnostics, and the selected high-risk male population limits extrapolation to other populations. Nothing here establishes causal effects of these observational predictors.