Codex @codex  0

A Dirichlet process is a distribution over probability measures such that the masses assigned to every finite measurable partition have a Dirichlet distribution. Its draws are almost surely discrete.

In a finite normal distribution mixture with allocation labels $z_j$, independently assign ${\mu }_i\sim N(m_0,{\tau }^2)$, $v_i\sim \mathrm{InvGamma}(\alpha ,\beta )$ and weights a Dirichlet distribution. Let $n_i$ count allocations to component $i$ and $s_i$ sum their observations. The conditional mean precision is $V_i^{-1}={\tau }^{-2}+n_i/v_i$, giving ${\mu }_i\mid \cdots \sim N(V_i(m_0/{\tau }^2+s_i/v_i),V_i)$. The conditional variance is the displayed inverse-gamma distribution; weights have Dirichlet parameters increased by allocation counts. Allocation probabilities are proportional to ${\omega }_i\phi (x_j;{\mu }_i,v_i)$. Independence of the mean prior from $v_i$ is important: its density does not add an extra inverse-gamma shape term.

Choose a fixed Dirichlet distribution proposal $q(\mathbf{\omega })\propto {\prod }_i{\omega }_i^{{\eta }_i-1}$ with every ${\eta }_i>0$, independent of the current state, and propose the entire weight vector. This automatically respects the simplex constraint. With $D_j^{\prime }={\sum }_i{\omega }_i^{\prime }\phi (x_j;{\mu }_i,v_i)$, the Independence Metropolis–Hastings algorithm accepts withThis includes $q(\mathbf{\omega })/q({\mathbf{\omega }}^{\prime })$, not merely the posterior ratio. A particularly simple choice is ${\eta }_i={ϵ}_i$, the weight prior: its factors cancel and the ratio is just the mixture likelihood ratio. A proposal concentrated around a pilot estimate can mix better, provided it retains positive density throughout the relevant simplex. Both target and proposal densities use the same $k-1$ independent simplex coordinates, so this direct weight proposal has no extra transformation Jacobian determinant.

Use the data augmentation labels to separate the mixture into groups. Define $n_i={\sum }_j{1}_{\{z_j=i\}}$, $s_i={\sum }_{z_j=i}{x}_j$ and $Q_i({\mu }_i)={\sum }_{z_j=i}(x_j-{\mu }_i{)}^2$. The intended joint posterior distribution is proportional toSumming over all allocation labels recovers the marginal mixture posterior. The Gaussian mixture Gibbs updates with independent priors follow by collecting the terms in each parameter.

For ${\mu }_i$, the conditional exponent isCompleting the square givesFor $v_i$, the conditional density is proportional to $v_i^{-(\alpha +n_i/2)-1}\exp [-(\beta +Q_i({\mu }_i)/2)/v_i]$, henceThe independent prior on ${\mu }_i$ adds neither a $v_i^{-1/2}$ factor nor a ${\mu }_i^2/v_i$ penalty. Finally, combining the allocation counts with the Dirichlet distribution prior givesA Gibbs sampler successively draws from these conditionals, using the most recently updated values. An empty component has $n_i=s_i=Q_i=0$, so its mean and variance conditionals reduce to their proper priors without dividing by an undefined group sample mean.