Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 35 / 1

Write $C$ for the binary response, $P$ for the partner count, and $Z$ for fully observed background variables. Let $R=(R_C,R_P)$ indicate which of $C,P$ are observed, with 1 meaning observed. For any realised pattern $r$, partition the data into $D_{\mathrm{obs}}(r)$ and $D_{\mathrm{mis}}(r)$.

Missing completely at random means that the missing-data mechanism does not depend on any of the measurements:Missing at random allows dependence on the data observed under the particular pattern, but no further dependence on its unobserved values:Thus a response probability depending on fully observed age or sex can satisfy missing at random. Dependence on an observed partner count can also be permissible for patterns in which that count is available. Missing not at random means that this equality fails: the probability of the pattern still depends on one of its missing measurements, after conditioning on the observed ones. The distinction concerns dependence on the unseen values, not whether missingness looks haphazard. These definitions can be applied within the target population $P>1$; unknown membership of that population must itself be handled when $P$ is missing.

Missing not at random is the plausible working classification. The suggested reluctance depends on the actual partner count and actual condom response, including the values that are withheld. Among people with the same observed age, sex and other recorded information, those with an unreported high count or an unreported negative response may still be less likely to answer.

Such dependence on unobserved values violates missing at random and therefore also missing completely at random. This is a substantive judgement rather than something that can be proved from the observed data alone. If all the dependence were explained by measured predictors, the mechanism could instead satisfy missing at random.

A complete-case analysis changes the mix of people contributing to the logistic regression. Let $A=1$ mean that both required variables are observed, and write $s_c(p,z)=\mathrm{Pr}(A=1\mid C=c,P=p,Z=z)$. Within $P>1$, Bayes theorem givesIf completeness depends differently on the two responses, and that selection also varies with partner count or the other predictors, the extra term changes the fitted regression coefficients. For example, people with high counts and poor condom use can be under-represented, weakening or otherwise distorting the apparent association. Dropping people with missing $P$ also excludes some whose membership of the $P>1$ target group is unknown.

The fitted conditional association among complete responders need not equal the target-population association. No direction of selection bias follows without specifying the response mechanism. Selection on predictors alone need not bias a correctly specified conditional logistic regression; it is the outcome-related selection described here that removes that reassurance.

Multiple imputation by chained equations specifies one conditional distribution for each incomplete variable. In general, let $V_1,\dots ,V_q$ be the incomplete variables, $Z$ the fully observed predictors, and ${\varphi }_j$ the parameters of a model $p_j(V_j\mid V_{-j},Z;{\varphi }_j)$. The notation $V_{-j}$ means all variables except $V_j$. The models should respect each variable's support and contain the terms needed by the planned analysis.

First choose suitable models and initialise only the missing entries with admissible provisional values. One sweep updates $j=1,\dots ,q$ in turn. At update $j$, fit its model to records whose $V_j$ is observed, using the latest filled-in predictors. DrawThe first draw represents parameter uncertainty through a Bayesian posterior; the second represents residual uncertainty through a posterior predictive distribution. Replace the missing entries with these draws, leaving every genuinely observed entry fixed. Repeat whole sweeps until the chain has settled, assessing traces, mixing and plausibility. Obtain $M$ completed datasets from independently initialised chains, or sufficiently separated draws after convergence. Both parameter and missing-value uncertainty must be propagated; repeated deterministic prediction is not multiple imputation.

Here use a logistic regression for $C$ conditional on $P$, age and sex. For $P$, use an appropriate discrete count model conditional on $C$, age and sex, such as a suitably checked negative binomial regression, allowing enough flexibility for skewness and nonlinear effects. Include interactions or transformations needed by the analysis in the imputation models. If condom use is structurally undefined for a record, preserve that applicability rule rather than treating it as ordinary missing data.

Impute the eligible survey information before discarding records solely because the observed $P$ is absent. In each completed dataset select its own $P>1$ records and fit the requested logistic regression. This allows a record with missing $P$ to contribute to the target group in some imputations and not others. Models must respect any known eligibility restrictions; if eligibility was independently known, use that information.

For a scalar regression coefficient, let ${\hat{Q}}_m$ be its estimate and $U_m$ its estimated variance in completed dataset $m$. Rubin's rules giveThe standard error is $\sqrt{T}$. Use the corresponding multiple-imputation confidence interval, rather than treating one completed dataset as fully observed. For vector coefficients replace the squared differences by outer products and $U_m$ by the covariance matrix. This combines within-imputation and between-imputation uncertainty.

Ordinary multiple imputation by chained equations relies on missing at random, conditional on the variables included in its models, together with adequate model specification and suitable conditional distributions. For Bayesian inference that ignores the response mechanism, the usual sufficient condition also includes distinct parameters for data and missingness models, with an appropriately separate prior.

The suspected dependence on withheld responses makes the usual missing-at-random assumption doubtful. An accurately fitted observed-data model cannot, on its own, establish what the nonresponders would have reported. Use substantive judgement and sensitivity analysis under plausible missing not at random departures, for example a pattern-mixture sensitivity analysis shifting the imputed condom log odds or partner-count distribution relative to the observed-data prediction. Report how the final association changes. Additional informative measurements may make missing at random more credible, but cannot verify it.

Treat the extra measurements as auxiliary variables for multiple imputation. Include alcohol consumption and disease history as predictors in both the $C$ and $P$ conditional distributions, together with age, sex and the other incomplete response. If these auxiliary variables themselves have missing entries, give them appropriate models and update them within the same multiple imputation by chained equations sweeps. Preserve observed entries throughout.

Useful auxiliary measurements can improve predictions, precision and the plausibility of missing at random. Their association with the missing responses reduces uncertainty in the imputed values; association with nonresponse may explain some selection that otherwise depends on unobserved values. This can reduce selection bias under the enlarged missing at random model and reduce between-imputation variance. It does not guarantee elimination of missing not at random dependence. They need not be added to the final substantive logistic regression merely because they help imputation: the final analysis should still estimate its intended association.