Past exam of the mathematics course of the University of Cambridge / 2013 / iii / Paper 34 / 4

A scoring rule in the reward convention gives $S(P,x)$ for announcing the probability distribution $P$ and then observing $x$. Under genuine belief $Q$, the expected score isA proper scoring rule satisfies $s(Q,Q)\ge s(P,Q)$ for every admissible $P,Q$. A strictly proper scoring rule additionally requires equality only when the distributions coincide:The expected scores must be well defined. A loss convention reverses the inequalities, but the question uses rewards.

Let $q$ be the genuine rain probability and $p$ the reported probability. The linear probability score has expected rewardThis is linear in $p$, so its optimal report and expected reward areAt $q=1/2$ every report scores $1/2$ on average. Truthful reporting scores $q^2+(1-q{)}^2$, strictly below the optimum for interior $q\ne 1/2$. Thus the rule is not proper: it rewards maximal confidence in the more likely outcome. The expected total over days is ${\sum }_t\max (q_t,1-q_t)$.

Let $q$ be the true density and $p$ the announced density. The advantage of truthful reporting for the logarithmic scoring rule isThe Kullback-Leibler divergence is nonnegative, with equality exactly when the densities agree almost everywhere. For instance, Jensen inequality gives ${\mathbb{E}}_q[\log (p/q)]\le \log {\mathbb{E}}_q[p/q]\le 0$; equality requires a constant likelihood ratio on the true support and no remaining mass outside it. ConsequentlyThis proves strict propriety when the expected log scores are well defined, for example with a finite true expected log density. A forecast assigning zero density to a set of positive true probability has score $-\infty$ there and cannot outperform the truth. The hint's strict $>0$ must be corrected to $\ge 0$ to allow its equality case.

Model $m$ gives the sequential forecast $f_{mt}(y_t)=p_m(y_t\mid y_1,\dots ,y_{t-1})$. The probability chain rule yieldsApplication of Bayes theorem makes the factors successive posterior predictive distributions, and their product is the Bayesian model evidence. Under conditionally independent sampling it isTaking logarithms gives the prequential log score identityTherefore the Bayes factor isThis solves the second half of part (d), which is missing from the TeX. Proper priors and finite positive evidences are needed; unrelated improper-prior normalizing constants do not cancel.

Let $M\in \{1,2\}$ be one fixed model indicator with equal prior probabilities, and put $w_{m,t}=\mathbb{P}(M=m\mid y_1,\dots ,y_{t-1})$. Within each model the parameters are already updated using the same past data. The Bayesian model averaging forecast is $f_t(y)={\sum }_m{w}_{m,t}{f}_{mt}(y)$. Upon observing $y_t$, Bayes theorem givesTaking the ratio cancels the denominator and gives the prescribed update because $f_{mt}(y_t)=e^{L_{mt}}$. Iteration yieldsThe weights are posterior model probabilities and their mixture is the full Bayesian predictive density. The indicator is fixed across days, rather than choosing a fresh model independently each morning.

Integrate each supplied predictive density to get its cumulative distribution function $F_t$, then form the sequential probability integral transformFor a correct continuous one-step conditional predictive model, $\mathbb{P}(U_t\le u\mid y_1,\dots ,y_{t-1})=u$. Iterating this identity shows that the $U_t$ are independent uniform variables. A histogram or quantile plot checks uniformity; serial plots and autocorrelations check for temporal structure left unexplained by the forecasts.

Also inspect empirical coverage of central prediction intervals, tail exceedances and interval widths, assessing calibration together with sharpness. These checks need only the supplied forecasts and observations. Compare chosen predictive discrepancy statistics with simulated uniform reference sequences. A total log score alone is a relative reward and does not provide a universal absolute goodness-of-fit threshold.