Past exam of the mathematics course of the University of Cambridge 2008 iii Paper 44 2 g Solution 2026-10-14
The gamma full conditional distribution has mean . The tower property and the beta distribution mean giveWriting and yieldsThus partial pooling pulls every local mean toward the overall sample mean. The common weight arises from the fixed shape-one mixing distribution; it is the Poisson–exponential posterior shrinkage formula.
Past exam of the mathematics course of the University of Cambridge 2008 iii Paper 47 5 a iii Solution Created 2026-10-03 Updated 2026-10-14
Choose an initial . At each iteration, evaluate and using and draw from the conditional multivariate normal distribution just derived. Next compute and drawFor example, generate and set . A draw of the conditional multivariate normal distribution can be formed as with a Cholesky decomposition and a vector of independent standard normal distributions.
This two-block Gibbs sampler alternates the full conditional distributions of the independent normal and inverse-gamma regression priors posterior. The conditional-integration argument proves that its invariant distribution is the joint posterior distribution. Under the usual convergence conditions, retain the pairs after an initial transient as the requested dependent posterior sample. The new variance update uses the newly drawn coefficients, rather than the previous coefficients.
Past exam of the mathematics course of the University of Cambridge 2008 iii Paper 47 5 a i Solution Created 2026-10-03 Updated 2026-10-14
A systematic Gibbs sampler updates the coordinates of successively from their full conditional distributions. At iteration , drawFor a single coordinate, if the current state has distribution , its untouched coordinates have the correct marginal law and the resampled coordinate has precisely the conditional law under . Integrating marginal times conditional therefore recovers . Each coordinate kernel preserves this invariant distribution, so their composition also preserves it. This proves the target of the Gibbs sampler; the whole systematic sweep need not be a reversible kernel.
Starting from a suitable state and assuming the required irreducibility and convergence conditions, discard an initial transient and retain subsequent states as a dependent sample from, or approximately from, . In finite-state problems irreducibility and aperiodicity suffice; in continuous-state problems the corresponding positive recurrence and ergodicity conditions must hold. Successive draws are generally dependent, so Monte Carlo uncertainty must account for that dependence.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 40 5 a Solution Created 2026-10-03 Updated 2026-10-14
For a target probability density function and a proposal distribution with density , the Metropolis–Hastings algorithm starts in the target support and repeats: propose , draw an independent uniform , and move to ifOtherwise retain . A zero reverse-proposal density makes the acceptance zero. Unknown multiplicative constants of cancel. For distinct states,which is symmetric in the states. Together with the probability of holding at the current state, this proves detailed balance and invariance of . Successive values form a dependent Markov chain; convergence from arbitrary initial states additionally needs the appropriate irreducibility and aperiodicity, not just this invariant-density identity.
The Gibbs sampler updates a coordinate using its full conditional distribution while keeping the others fixed. For coordinate , write and . Its proposal replaces by . The MH ratio, interpreted on this one-coordinate proposal space, isThus every Gibbs coordinate update is an MH update with acceptance one. A random choice of coordinate gives a mixture of these invariant kernels; a deterministic sweep gives their composition. Both preserve . The latter composition need not be reversible, so the special-case statement is about the coordinate kernels, not a claim that every entire systematic sweep is one reversible MH kernel.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 40 5 b Solution 2026-10-14
At , complete the square:For fixed , all factors not involving enter its conditional normalizing constant. Interchanging the roles of the coordinates givesThese are the full conditional distributions of the standard bivariate normal distribution with correlation .
A systematic Gibbs sampler starts at a chosen and, at each sweep, draws independent standard normals and setsThe second update uses the newly drawn . Both coordinate updates preserve the target by part (a), hence so does the sweep. The Gaussian two-coordinate Gibbs recursion is explicitlyan AR(1) recursion with innovation variance . Its stationary variance is one, and its correlation at sweep lag is , displaying the dependence of the sample.
The printed general- expression has a separate normalization defect: for its integral is , not one, since the exponent has covariance , whose normalizing denominator is . Retaining that exponent requires in the prefactor. The requested target is already normalized, so this defect does not alter the conditional distributions above.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 40 6 d Solution Created 2026-10-03 Updated 2026-10-14
Both EM algorithm and data augmentation exploit a latent completion , but they use it differently. EM takes a conditional expectation of the complete-data log-likelihood and then maximizes over the parameter, producing a deterministic sequence of point estimates when the M-step is uniquely specified.
In Bayesian data augmentation, choose a prior distribution and targetAlternate the Gibbs sampler updatesThese are stochastic draws from full conditional distributions, not an expectation followed by maximization. Their invariant law is the augmented joint posterior, and discarding gives posterior inference for , including uncertainty rather than only an MLE. A sampled completion followed by maximization would instead be a stochastic EM variant. Individual likelihood values along a data-augmentation chain need not increase: the chain is designed to sample a posterior, not to ascend the likelihood.
Past exam of the mathematics course of the University of Cambridge 2010 iii Paper 33 5 Solution Created 2026-10-03 Updated 2026-10-20
The Metropolis–Hastings algorithm proposes and moves from the current state with acceptance probabilityOn rejection it retains . The target need only be known up to its normalizing constant, but it must define a proper probability distribution. The accepted off-diagonal probability flow iswhich is symmetric in and proves detailed balance. This permits considerable flexibility in proposals, including joint moves when a posterior distribution has strongly correlated parameters, but requires proposal tuning and can waste computation on rejections.
A Gibbs sampler instead updates a coordinate or block from its exact full conditional distribution, holding the other coordinates fixed. This is a special Metropolis–Hastings algorithm update with acceptance probability one: the joint density factors into the unchanged marginal density and the proposed conditional density, which cancel in the acceptance ratio. The coordinate update preserves the target; composing such updates therefore preserves it too, although an entire deterministic sweep need not itself be reversible.
Use Gibbs sampling when the conditional laws are easy to simulate and suitably chosen blocks mix well. Scalar updates can mix poorly under strong posterior distribution correlation, whereas a blocked Gibbs sampler can alleviate that dependence. Prefer a flexible Metropolis proposal when conditional draws are difficult or a well-tuned joint proposal is more effective. Variations include random or systematic coordinate scans, random-walk Metropolis algorithms, independence proposals, and using Metropolis updates for difficult blocks inside a Gibbs scheme. Both methods generate dependent draws; usable long-run estimates require an appropriate irreducible, aperiodic chain and assessment of mixing, not merely a formal update rule.
For the change-point calculation, put andMultiplication of the Gaussian likelihood function by the specified prior distribution gives the formal posterior distribution kernel, with respect to and counting measure on ,
There is an empty component under an improper prior at . The sum is independent of , whose prior distribution is flat on all of . For every fixed finite and , the kernel is a strictly positive constant in , henceThe configuration has positive prior probability . Integrating over, for example, a bounded interval of and already makes the joint normalization infinite. Thus the printed prior distribution does not produce a posterior probability distribution. This failure is present for every data set, not just for a special arrangement of observations.
The formal full conditionals expose exactly where the requested sampler breaks. For , completing the two mean squares givesThe variance kernel is that of when , and the discrete conditional probabilities are proportional to . At the conditional kernel for is constant and cannot be normalized; it is not a normal distribution with an infinite variance. Therefore no valid Gibbs sampler, and no value of , exists for the model as printed. A chain that simply substitutes an arbitrary distribution for this missing conditional would be sampling a different model.
Here is a fully specified proper-prior repair retaining the no-change configuration. Choose fixed hyperparametersindependently, with . Its Gaussian change-point posterior with proper priors isIt is normalizable even for degenerate data: the likelihood function is at most , and its integral against the variance prior distribution is bounded byThe normal mean priors integrate to one, and the normalizing constant is positive. This proves posterior propriety for the repaired model.
Completing the conditional squares gives the Gaussian change-point Gibbs updates. DefineThe four updates areAt , , giving its proper prior distribution unchanged. Starting from any finite means, positive variance and allowed , cycle through these four updates using the latest values. The sum-of-squares terms can be evaluated from cumulative data sums; normalize the discrete log weights after subtracting their maximum to avoid numerical underflow. The proper positive conditionals allow movement among all split values and throughout the continuous parameter support.
Under this specified repair, is the posterior probability of no change within the observed sequence: all data points belong to the first mean regime. For retained draws after initialization has ceased to affect the estimates, its Markov chain Monte Carlo estimator isAlternatively average the displayed conditional probability of at the sampled continuous parameters, giving an estimate by Rao-Blackwellization. Autocorrelation must be accounted for in its Monte Carlo uncertainty. These are probabilities and estimators for the explicitly repaired model; the original unnormalizable kernel supplies neither.
Past exam of the mathematics course of the University of Cambridge 2012 iii Paper 38 4 Solution Created 2026-10-03 Updated 2026-10-07
A Gibbs sampler updates one coordinate by drawing from its full conditional distribution, leaving the others unchanged and using their current values. Each coordinate kernel preserves the target joint law : integrate first over the unchanged coordinates, then over the old coordinate, and finally draw the new coordinate from the same conditional distribution. The resulting joint law is again . In the discrete notation, if two states share the remaining coordinates,which is symmetric in . Thus a coordinate update satisfies detailed balance. A systematic full sweep has kernel and remains invariant because each factor preserves . A fixed-probability random-scan mixture also preserves ; the mixture is reversible, whereas a systematic composition need not be.
In the autologistic binary-image model, count each neighboring pair once. Changing one pixel from zero to one raises by one and by the number of neighboring ones. The ratio of its two conditional weights is . ThereforeFor single-site Gibbs sampling, select a pixel, compute its current neighbor sum and replace it by a draw from a Bernoulli distribution with this probability. One may use random scans or complete ordered sweeps. The normalizing constant is not needed. For finite , every conditional probability lies strictly between zero and one; the finite chain is irreducible and has self-transitions, so its stationary law is unique and iterating converges to it.
For checkerboard Gibbs sampling, no edge joins two black pixels or two white pixels. Conditional on the whites, the black-pixel law factorizes:Draw all black pixels independently with these probabilities, then draw all white pixels independently conditional on the newly drawn blacks. Each color block is a full conditional update, so the two-block sweep preserves the joint target. Pixels of the same color can be updated together because their conditional distribution factorizes.
For the noisy image, the log likelihood contributes . Since , dropping terms independent of givesThe Gaussian-noise posterior for an autologistic image therefore has conditional probabilitiesUse this probability in both single-pixel and sequential color-block schemes. The independent observation factors change only the local field, so conditional independence within each color is retained. The same invariance proof then targets the posterior distribution rather than the prior.
Past exam of the mathematics course of the University of Cambridge 2013 iii Paper 29 3 Solution Created 2026-10-03 Updated 2026-10-07
For the Box-Muller transform, draw independent from the uniform distribution on and setThe radial density is for , with an independent uniform angle. The polar-to-Cartesian Jacobian determinant is , so the joint density of isThis factorization proves that both outputs have the standard normal distribution and are independent. Repeating the Box-Muller transform with fresh independent uniform pairs gives independent normal outputs; discard one extra output if the desired sample size is odd.
For the prescribed binary probabilities, generate independent standard normal values in this way and useThe standard normal distribution function gives , and independence is preserved because each threshold uses a different independent normal value.
For the probit regression posterior, introduce latent variablesGiven , these are independent variables, and normal symmetry gives . Thus this data augmentation has exactly the observed binary likelihood. With the specified normal prior distribution, the augmented joint posterior is proportional toThe latent-normal Gibbs sampler for probit regression alternates two blocks. First, given the current , draw each independently from its truncated normal distribution, namely restricted to the sign fixed by . One exact method is to use the Box-Muller transform for , form , and reject until the sign is correct. The probability of success is positive at every finite parameter value, so the method is valid, although it can be slow for a rare sign.
For direct sign-truncated normal sampling, let , and draw . An inverse transform sampling implementation isUse suitable tail or survival-function evaluations when floating-point probabilities approach zero or one; the rejection construction remains a valid alternative.
Second, completing the square in yields its full conditional distribution:Generate a fresh standard normal value by the Box-Muller transform, multiply by the conditional standard deviation, and add the conditional mean. Start from any finite , alternate these steps, discard an initial transient and use the retained values to approximate its posterior distribution. These are dependent Markov chain Monte Carlo samples, rather than independent posterior draws. Each block is an exact Gibbs sampling update for the augmented posterior, whose marginal in is the requested posterior.
Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 37 5 d 1 Solution Created 2026-10-03 Updated 2026-10-06
The algorithm is a Random-scan Gibbs sampler. Start at a point where the target is positive, with a proper normalizing integral and well-defined full conditional distributions. Its state is the entire coordinate vector. Choosing the initial point fixes the initial law; it need not already be the invariant law. The transition rule below preserves the target irrespective of this choice.
Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 37 5 d 2 Solution Created 2026-10-03 Updated 2026-10-06
At each transition, choose a coordinate uniformly and independently of the previous coordinate choices. Draw a fresh value from that coordinate's full conditional distribution given the current remaining coordinates, and keep all other coordinates fixed. This supplies the random-selection and conditional-draw operations lost in the TeX transcription. The update depends only on the current vector and fresh randomness, which proves the Markov property. The printed mathematical requests are the kernel and its detailed balance, answered in the following two subsections.
Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 37 6 i e Solution Created 2026-10-03 Updated 2026-10-06
Fix any configurations and . List the sites where they differ and update those sites in turn to their target signs. Every specified site has positive probability of being selected. At every intermediate configuration, both values of its full conditional distribution have positive probability, since all weights are finite and strictly positive. The finite prescribed sequence therefore has positive probability and reaches .
Thus every configuration is accessible from every other configuration, so the sampler is an irreducible Markov chain, or equivalently phi-irreducible with counting measure. If , a single update that keeps its selected spin unchanged also has positive probability. This simultaneously proves positive self-transition probability and aperiodicity, including . The sign of finite does not alter the accessibility argument.
Past exam of the mathematics course of the University of Cambridge 2017 iii Paper 216 1 a Solution Created 2026-10-03 Updated 2026-10-06
Put and . Dropping factors independent of , the multinomial likelihood and uniform prior give the posterior densityMultiplying this by the stipulated binomial distribution for the latent variable cancels the factor :Consequently the two full conditional distributions areStart with , sample from its binomial distribution, then sample a new from its Beta distribution, and repeat. The binomial distribution can be sampled by adding independent Bernoulli random variables. For the Beta distribution, take independent and and return ; integer-shape gamma distributions are sums of independent unit-rate exponential distributions. Both shapes are positive even when some observed counts vanish.
Each Gibbs sampler update preserves the augmented posterior distribution, so its marginal distribution is the required posterior distribution. The positive full conditional distributions on the interior allow exploration of the entire support. These iterates are generally dependent; they are not the independent exact draws constructed in the later parts.
Past exam of the mathematics course of the University of Cambridge 2017 iii Paper 216 3 a Solution Created 2026-10-03 Updated 2026-10-06
Use the shape-rate convention for every gamma distribution, with . Let and . The Poisson distribution likelihood and the Gamma–Poisson hierarchical model giveAll factors involving the coordinate being updated must be retained, including from the conditional gamma distributions. The full conditional distributions areFor a blocked Gibbs sampler, draw from the second full conditional distribution, then redraw all the independently from the first ones. Repeating these two steps preserves the joint posterior distribution; observing after each complete sweep gives the Markov chain used in the next part. Standard gamma distribution sampling works for noninteger as well as integer shapes. The positive observation-period convention is used for the subsequent geometric drift condition.
Past exam of the mathematics course of the University of Cambridge 2017 iii Paper 216 3 b Solution Created 2026-10-03 Updated 2026-10-06
Take observation periods to have positive lengths , and let be the Markov kernel of a sweep which first draws given the current and then draws the new vector . Write . Conditional on , independence and the gamma distribution mean and variance giveAveraging over the full conditional distribution of preserves this bound, uniformly in the current state. Therefore . This is the mechanism in bounded conditional moments imply a geometric drift.
Choose , , and . Outside , ; inside , . HenceFor completeness, is a small set, even though it approaches the boundary of the positive orthant. Choose small enough that . On , . The conditional gamma distribution of has fixed shape and rate in a compact positive interval. Its probability density function therefore has a positive common lower bound on . For those , the product probability density function of similarly has a positive common lower bound on , since every shape is positive and every rate lies in a compact positive interval. Integrating over gives on , where is the uniform distribution on and .
The transition probability density function is positive throughout the positive orthant, giving an irreducible Markov chain. Since , the box minorization includes starts in the same box, which gives an aperiodic Markov chain. The proper prior distributions and bounded Poisson distribution likelihood have positive finite evidence, so the invariant posterior distribution is proper. Together with the geometric drift condition, these hypotheses imply geometric ergodicity. In particular, the argument proves the required drift rather than assuming that every Gibbs sampler is geometrically ergodic.
The positive-period convention matters. If zero lengths are allowed, the printed assertion can fail: take , , . Then and , soThis example has a proper posterior distribution but cannot satisfy a finite quadratic geometric drift condition. Thus the proof establishes the intended claim for positive observation periods, not an unrestricted extension to zero exposure.
Past exam of the mathematics course of the University of Cambridge 2017 iii Paper 216 5 a Solution Created 2026-10-03 Updated 2026-10-06
The most plausible cause is strong dependence between the latent variables and covariance hyperparameters in their joint posterior distribution. In the usual conditionally independent Gaussian process classification model, put , , and . The probit regression likelihood is , while has a multivariate normal distribution with covariance matrix .
For distinct inputs and positive length scales, is positive definite. Its Gaussian probability density function contributesThe unit-rate exponential prior distribution on therefore givesIts scale is tightly tied to the current magnitude and shape of . Changing changes the eigenvectors and eigenvalues of its covariance matrix; a latent vector typical under the old covariance may be very atypical under a substantially different one. Holding fixed can thus restrict the range of a hyperparameter update, and holding the hyperparameters fixed restricts the next latent-function update. Even exact sampling of each full conditional distribution can move slowly along a narrow ridge of the joint posterior distribution.
Binary observations provide limited information about the absolute size of large correctly signed latent values. This can leave substantial scale uncertainty and strengthen the dependence. Long correlation lengths can also make the covariance matrix nearly singular and the latent coordinates highly dependent. These are plausible explanations for a large mixing time, not proofs that every data set causes slow convergence. A blocked Gibbs sampler removes dependence within its chosen blocks, but not dependence between them. Repeated identical inputs should be represented by one shared latent value; otherwise the nominal Gaussian matrix is singular and the displayed inverse formula must be replaced by a representation on its support.
Past exam of the mathematics course of the University of Cambridge 2019 iii Paper 219 4 d Solution 2026-10-03
Every move can be drawn from a full conditional distribution, producing a Gibbs sampler with acceptance probability one. Write and . First update independentlyand thenLet have rows and . Update the linear regression coefficients jointly byand updateFinally, the flat positive variance priors give the following full conditionals, each an inverse-gamma distribution:A systematic sweep in the displayed order, using the newly sampled values immediately, defines the chain. The shapes are positive for ; full column rank of is also required.
Past exam of the mathematics course of the University of Cambridge 2022 iii Paper 219 3 c Solution 2026-09-28
A Gibbs sampler draws each variable from its full conditional distribution, so every proposed update is accepted. At the start of a sweep define , , andFirst update the reddenings independently asRecompute and , then make the Gaussian updatesandWiththe remaining full conditionals, in the parameterization given in the question, are the scaled inverse chi-squared lawsRepeating these updates in the displayed order gives a complete Gibbs sweep. The formulas assume and nondegenerate sampled predictors; posterior propriety must be checked because the hyperpriors are improper.
An unknown variance requires a joint prior distribution for . The analysis is not determined until a prior for the unknown variance, and its relation to the coefficients, is specified. A maximum-likelihood estimate substituted for does not by itself account for its posterior uncertainty.
For example, retain the independent coefficient prior distributions from part (b) and independently assign an inverse-gamma distribution with shape and scale , whose probability density function is proportional to . Then the joint posterior density is proportional toConditional on , the two coefficient posterior distributions are still the independent normal distributions in part (b), with substituted for . The other full conditional distribution isThe marginal coefficient posterior distribution requires marginalization over ; it is generally nonnormal and the two coefficients need not remain independent. In particular, the posterior cumulative distribution function of the requested mean becomesIts shortest 95% credible interval must be found from this marginal posterior distribution, for example by numerical integration or posterior simulation. This is normal regression with independent coefficient and variance priors.
A different, convenient choice is a normal-inverse-gamma prior, in which the conditional coefficient prior covariance matrix is proportional to . That choice gives a multivariate Student's t-distribution after marginalization, so the interval uses its univariate Student's t-distribution quantiles. It changes the coefficient prior specification; a Student interval does not follow automatically from retaining the fixed- priors in part (b).
For and , a scale-invariant prior induces the Haldane prior on . Integrating out each local mean gives . Thus for . The gamma full conditional distribution has mean , so the tower property gives , equal to the displayed partial pooling formula. For , the posterior is improper and this expectation is undefined.