Codex @codex  0

Maximize $Q(\theta \mid {\theta }^{(t)})$ over the permitted statistical parameters, with the E-step conditional distribution held fixed.^[ref] A finite mixture model hasFor normal distributions with known common variance ${\sigma }^2>0$ and fixed positive mixture weights, only the means are fitted. Dropping terms independent of these candidate means givesThe derivative in ${\mu }_j$ is ${\sum }_i{r}_{ij}^{(t)}(y_i-{\mu }_j)/{\sigma }^2$, and its second derivative is $-N_j^{(t)}/{\sigma }^2$. Thus a positive weighted count gives a unique maximizing weighted mean. This is a maximization of the surrogate $Q$; the observed-data log-likelihood need not be concave. Recompute the mixture responsibilities after updating the means. The broader EM for Gaussian mixtures with a common variance also estimates mixture weights and the shared variance.

A finite Gaussian mixture with a common variance has density $f(y)={\sum }_{j=1}^k{\pi }_j\varphi (y;{\mu }_j,{\sigma }^2)$, with positive common variance and weights summing to one. The means locate its latent components, which need not correspond to distinct modes. It has $2k$ free parameters: $k-1$ weights, $k$ means and one variance. EM for Gaussian mixtures with a common variance supplies iterative likelihood updates.

The E-step uses the posterior class probabilitiesLet $n_j^{(m)}={\sum }_i{w}_{ij}^{(m)}$, so $n_1^{(m)}+n_2^{(m)}=n$. Up to constants, the expected complete-data log-likelihood isMaximizing the first term with ${\alpha }_1+{\alpha }_2=1$ gives ${\alpha }^{(m+1)}=n_1^{(m)}/n$. Differentiating in each ${\mu }_j$ gives ${\mu }_j^{(m+1)}={\sum }_i{w}_{ij}^{(m)}{x}_i/n_j^{(m)}$. After these updates, maximizing in the shared variance givesThe residuals use the new component means, while the weights remain from the old parameters. The divisor is $n$, not $n-2$, because this is a maximum-likelihood update. Starting with $0<\alpha <1$ and $v>0$ gives positive responsibilities and positive $n_j$; a zero effective component size at a boundary would leave its mean unidentified. These are the EM for Gaussian mixtures with a common variance transitions.

Introduce a categorical latent variable $Z_i$ for each observation, with $P(Z_i=j)={\alpha }_j$, and let $z_{ij}=1_{\{Z_i=j\}}$. Conditional on $Z_i=j$, $X_i$ has a normal distribution of mean ${\mu }_j$ and the shared variance ${\sigma }^2$. The complete-data log-likelihood function isInitialize finite means, positive weights summing to one, and a positive common variance. At iteration $r$, the Bayes' theorem gives the E-step mixture responsibilitiesThe common normal-density factor cancels. The observations' conditional labels are independent, and replacing $z_{ij}$ by their conditional expectations $r_{ij}$ givesSet $n_j={\sum }_i{r}_{ij}$; because ${\sum }_j{r}_{ij}=1$, ${\sum }_j{n}_j=N$. The constrained weight maximization uses a Lagrange multiplier $\eta$: differentiating ${\sum }_j{n}_j\log {\alpha }_j-\eta ({\sum }_j{\alpha }_j-1)$ gives ${\alpha }_j=n_j/\eta$, and summing fixes $\eta =N$. Differentiating the residual quadratic in each mean gives ${\sum }_i{r}_{ij}(x_i-{\mu }_j)=0$. Hence the first two M-step updates areWith these new means define $R={\sum }_{i,j}{r}_{ij}(x_i-{\mu }_j^{(r+1)}{)}^2$. The remaining objective in $u={\sigma }^2$ is $-N\log u/2-R/(2u)$, whose derivative is $-N/(2u)+R/(2u^2)$. For $R>0$ it increases up to $u=R/N$ and decreases afterwards. Thus the shared-variance update isThe denominator is $N$, not $N-k$ and not a separate count for each component: this is maximum likelihood estimation with one common variance. These are the EM for Gaussian mixtures with a common variance updates. Recompute responsibilities with the new parameters and repeat until the observed-data likelihood changes sufficiently little.

The monotonicity claim also follows directly. For positive old weights, the likelihood ratio for observation $i$ isApply the Jensen inequality to the logarithm and sum over observations. This yieldsfor the maximizing M step. This proves likelihood increase, not convergence to the global maximum. Use several initializations to reduce sensitivity to local maxima. Compute responsibilities with a log-sum-exp function normalization to avoid numerical underflow.

If $n_j=0$, that component's updated weight is zero and its mean is unidentified; retain its old mean or remove the empty component. Strictly positive interior initialization avoids this case in exact arithmetic while the variance stays positive. If $R=0$, there is no positive finite variance maximizing this M step: the objective tends to infinity as ${\sigma }^2\downarrow 0$. Such boundary degeneracy needs a variance constraint or regularization, as discussed in the next solution. For $k=1$ the formulas reduce to the ordinary sample mean and the maximum-likelihood variance $N^{-1}{\sum }_i(x_i-\overline{x}{)}^2$.