Topics (384k) Articles (398k) Users (410) Discussions (242) Comments (383) Files (13k) New article
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 6 iii Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
For observed data and missing data , let be the complete-data model. Starting at , the expectation-maximization algorithm alternatesThe E-step computes the conditional expectation of the complete-data log-likelihood, using the current parameter for the missing-data conditional distribution. The M-step maximizes that expected log-likelihood over a new parameter. Maximizing the likelihood of imputed mean data is equivalent only when the complete-data log-likelihood has the necessary linear dependence on the missing sufficient statistics.
For the monotonicity argument, let . Jensen inequality givesTherefore an M-step increasing cannot decrease the observed-data likelihood. Standard support and integrability conditions are understood. EM need not find a global maximum for a general model; its behavior depends on the objective and starting point.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 6 ii Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Let be the unobserved part of the fourth count associated with the component of probability . Its complementary subcount has probability . Conditional on the observed fourth count and a parameter value,The complete-data log-likelihood now has the simple formThe constant term can depend on the completed counts, but not on . The split therefore converts the troublesome term into a binomial-like maximization problem. Since is unobserved, it should be averaged conditionally in an EM algorithm, not guessed or treated as observed.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 6 i Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
The parameter range is . Omitting data-only constants from the multinomial distribution likelihood, its log isFor an interior maximizer, the likelihood score must vanish:Multiply by and collect powers to obtainThe likelihood is concave: its second derivative in the interior isTherefore an admissible interior root is the unique maximum. Endpoints must also be handled if some counts vanish. If and , both endpoints have zero likelihood and the root is interior. If the maximum is at zero then , and zero also satisfies the displayed polynomial; a maximum at one requires , and one likewise satisfies it. Multiplying the score can introduce other endpoint roots, so not every polynomial root is an MLE: evaluate the actual likelihood on the feasible candidates.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 5 vi Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Under the reference convention , the hypothesis becomes absence of the free group contrast . Introduce an inclusion indicator : set when , and use its stated proper normal prior when . Assign positive prior probabilities to those models, and retain in both.
A birth draws from a proposal probability density function and appends it as ; the reverse death deletes it. The Jacobian is one. At fixed , the likelihood ratio isThus the birth acceptance ratio before capping iswhere is the normalized prior. Again, taking cancels that factor. Use the reciprocal death rule and ordinary within-model parameter updates.
After warm-up, the Markov chain Monte Carlo occupation fraction estimates the desired probability:This requires the chain to explore both models and satisfy its usual ergodic convergence conditions. Under the original continuous normal prior alone, the equality event has posterior probability zero; the explicit model indicator supplies the necessary prior atom. If intercept selection from part (v) is also performed, use two indicators and count all sampled states with , regardless of the intercept indicator.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 5 v Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Introduce a model indicator . In , set , hence , and retain the two group effects. In , include with the prior induced by the gamma baseline. Choose positive prior model probabilities summing to one; the supplied within-model parameter priors alone do not specify these probabilities.
Let , use the same proper prior in both models, and denote their likelihoods by and . The unnormalized joint model targets areFor a birth move, draw an auxiliary from a positive probability density function and map to . The reverse death move simply deletes . This is a dimension-matching bijection with absolute Jacobian one. If the birth and death selection probabilities are and , respectively, the birth and death moves for Bayesian variable selection rule givesThe reverse acceptance is the reciprocal ratio, capped at one, evaluated at the current . Choosing cancels the added-parameter prior; a data-informed proposal may mix better but must retain its proposal factor. Within each model, also update its continuous parameters to ensure exploration.
For an explicit likelihood ratio, with and from part (ii),This supplies a complete reversible-jump Markov chain Monte Carlo move. Its acceptance probabilities enforce detailed balance between the two dimensions. The lower-dimensional state is a separate point-mass model, not a zero-probability equality test inside the continuous full-model prior.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 5 iv Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
The full conditional log probability density functions of the effects are, up to constants,These are Gaussian-prior Poisson log-effect conditional probability density functions. The exponential likelihood term prevents a standard normal distribution conjugate update. A Metropolis-within-Gibbs algorithm conveniently updates one effect at a time without computing its conditional normalizing constant.
For example, propose with , and accept withUse an analogous normal distribution random-walk proposal for . The proposal is symmetric and lives on the appropriate unrestricted real parameter space, so no proposal ratio is needed. It makes local moves compatible with a unimodal conditional probability density function. Tune the positive scale in a pilot or freeze it after warm-up; a scale far too large causes rejection and a scale far too small moves slowly.
A useful scale can be inferred from the conditional curvature: , with the analogous formula for . A normal proposal near the conditional mode with variance approximately is another sensible choice, but an asymmetric or state-dependent proposal must include its full reverse-to-forward proposal-density ratio. Metropolis–Hastings is convenient, not logically mandatory; these concave conditionals also admit other specialized samplers.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 5 iii Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
An exact Gibbs sampler update is available despite the non-Gaussian intercept: draw from the conditional gamma distribution law in part (ii), then setTogether with conditional updates of the two remaining effects, this targets the required joint posterior in the log-intercept coordinates. The log-gamma prior for a Poisson log-intercept includes the change-of-variable factor: with ,Using instead would omit the Jacobian and give the wrong conditional law for .
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 5 ii Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Use the standard interpretation that the counts are conditionally independent given the parameters, with mutually independent stated priors. Let , , and defineUnder the reference constraints, the Poisson distribution likelihood, up to data-only factors, isMultiplying by the shape-rate gamma prior gives the Poisson-gamma conjugacy with unequal exposuresHere is a rate, matching the probability density function convention on the PDF's first page. The factor represents the number of replicates per cell, not the total count.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 5 i Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
The linear predictor is unchanged by , , . Without constraints, different parameter triples therefore give the same rates and likelihood. Identifiability requires choosing one representative of this redundancy.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 4 vi Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Antithetic variates preserve each draw's marginal law but couple two estimates negatively. For an average of identically distributed ,Negative covariance improves on two independent evaluations at the same computational budget.
Here generate independent and pair with . Both are uniform on . The antithetic Cauchy-tail integration on a finite interval estimator isIt is unbiased. Put . This decreases, whereas increases. For an independent copy of , the covariance identity givesEach product is nonpositive, and for it is strictly negative off the diagonal. Thereforewhich is the variance using independent evaluations in part (v). Thus the comparison accounts for the doubled number of probability density function evaluations in each pair.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 4 v Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Take , so the proposal is the uniform distribution on , with probability density function . Symmetry of the standard Cauchy distribution givesThus the proposed expression is exactly the unbiased importance sampling estimator of this complementary integral:It has finite variance because is bounded on . Explicitly,These follow from the antiderivatives of and . The target itself is . At it is exactly without simulation; for negative the printed uniform-interval construction must be changed.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 4 iv Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
For the standard Cauchy distribution tail, use the indicatorThen is the desired probability, and the ordinary importance sampling estimator is . The proposal must cover the nonzero integrand, namely the upper tail. A proposal restricted to cannot directly estimate this indicator integral; part (v) instead uses a complementary bounded-interval identity.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 4 iii Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Set . Independence gives , while part (i) gives . Its second moment isConsequentlyThe variance is finite exactly when the displayed second-moment integral is finite, assuming exists. An infinite integral means infinite variance, not failure of the change-of-density identity.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 4 ii Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Write and , where are known nonnegative kernels. If both constants are known, the ordinary weight is . If their ratio is unknown, use self-normalized importance sampling:Indeed, under the normalized proposal law,The strong law of large numbers makes their empirical ratio converge to , provided the numerator is absolutely integrable and . Multiplicative constants cancel, whether the unknown one belongs to the target, proposal, or both. Unlike the estimator in part (i), this ratio is generally biased at finite sample size. Exact sampling from is still presumed; knowing only an unnormalized proposal does not itself supply a sampler.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 4 i Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Assume the draws are independent from , that the target-weighted integrand is covered by its support, and that . Define the importance weight . The change-of-density identity isTherefore importance sampling uses the unbiased estimatorThe strong law of large numbers gives consistency. A finite second moment of the weighted integrand additionally gives finite variance and a usual independent-sample central limit theorem. A useful proposal places enough probability where is large and avoids tiny proposal probability density function there; common support alone does not guarantee finite variance.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 3 iv Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
For , use the unnormalized normal shape . The ratio-of-uniforms method samples uniformly fromIts boundary is . The largest possible occurs at and equals , giving the normal ratio-of-uniforms envelope , .
Use independent uniform pairs to propose and . Accept precisely whenTo verify its distribution, change coordinates from to , with and Jacobian determinant . The accepted point is uniform on , and the marginal probability density function of is proportional toThus the result is . The region has positive finite area , and the acceptance probability is . The case requires simply returning zero.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 3 iii Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Use uniform-envelope rejection sampling for a beta distribution. Put for and choose . For , differentiation of gives the mode , soIf or , take ; this also covers . Draw and accept it ifOtherwise continue with the next independent pair. The probability of acceptance is , so the procedure terminates almost surely. Conditional on acceptance, the probability density function is proportional to , hence is exactly the beta distribution of parameters . The restriction guarantees the bounded uniform envelope.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 3 ii Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
For rate , returnFor , , so has the exponential distribution of rate . This is inverse transform sampling; using is equivalent. Uniform draws at exact endpoints can be excluded, since those events have probability zero under the ideal continuous law.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 3 i Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Treat the supplied uniform variates as independent draws, as required for Monte Carlo simulation. Use of them and returnEach indicator has a Bernoulli distribution of parameter , and the indicators are independent. For any subset of successes the probability is ; there are such subsets. Thus , the binomial distribution. The construction includes the cases and .
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 40 2 c Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
A constant posterior variance must be a fixed point of the recursion in part (b). Multiplying by its positive denominator givesConversely, if a nonnegative solves this equation and the filter is initialized with , the recursion preserves at every step. The fixed-point equation alone does not force an arbitrary initial covariance to be constant.
For , the admissible root and its steady-state local-level Kalman gain areThe state estimate then obeys exponential smoothing:Iteration makes the weights explicit:Thus older observations receive geometrically decreasing weights. Consistently with part (a), the innovations variance is and the differenced MA coefficient is . The degeneracies and give gain zero and gain one respectively when the denominator is positive.
Pinned article: Introduction to the OurBigBook Project
Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
Intro to OurBigBook
. Source. We have two killer features:
- topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculusArticles of different users are sorted by upvote within each article page. This feature is a bit like:
- a Wikipedia where each user can have their own version of each article
- a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.Figure 1. Screenshot of the "Derivative" topic page. View it live at: ourbigbook.com/go/topic/derivativeVideo 2. OurBigBook Web topics demo. Source. - local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
- to OurBigBook.com to get awesome multi-user features like topics and likes
- as HTML files to a static website, which you can host yourself for free on many external providers like GitHub Pages, and remain in full control
Figure 2. You can publish local OurBigBook lightweight markup files to either OurBigBook.com or as a static website.Figure 3. Visual Studio Code extension installation.Figure 5. . You can also edit articles on the Web editor without installing anything locally. Video 3. Edit locally and publish demo. Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension. - Infinitely deep tables of contents:
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact





