Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 207 / 3 / c

Let $N_m=|\{i:c_i=m\}|$. The label likelihood for the mixture weights is proportional to ${\prod }_{m=1}^3{\pi }_m^{N_m}$. Multiplication by the $\mathrm{Dirichlet}({\alpha }_1,{\alpha }_2,{\alpha }_3)$ density and Dirichlet-multinomial conjugacy givesConditional on the labels, the observations and component means provide no further information about $\pi$.

For component $m$, stack its $N_m$ assigned observations as $y_1^{(m)},\dots ,y_{N_m}^{(m)}\in {\mathbb{R}}^n$. The prior is $f_m\sim N_n(0,K)$ and each assigned vector is conditionally $N_n(f_m,{\sigma }^2{I}_n)$. Normal-normal conjugacy giveswhereIf $N_m=0$, this reduces to the prior $N_n(0,K)$.

Bayes theorem turns the categorical distribution prior probabilities and the component multivariate normal densities intoThese probabilities define the label update in the Gibbs sampler.

A Dirichlet process mixture model avoids fixing the number of occupied functions. Let $G_0=N_n(0,K)$ be the finite-dimensional Gaussian process law on the common input grid and specifyA draw from a Dirichlet process is almost surely discrete, so several $f_i$ coincide and thereby form clusters. The number of occupied clusters is random and can grow with the data.