Identical-mean EM trap

In the Known-variance Gaussian mixture EM example, initialize both means at the same value $m$. The mixture responsibilities are then $1/2$ for every observation. Each M-step sets both means to the sample mean, zero here; the next E-step again gives $1/2$. Thus $(0,0)$ is a fixed point with log-likelihood $-14.593631199$, despite the better fit found from $(-1,1)$.

One can check the failure without trusting an iteration plot. Separate the means as $(-a,a)$ and use the hyperbolic cosine identity:The Taylor expansion uses ${\sum }_i{y}_i^2=18.16$. Small nonzero separation therefore improves the log-likelihood, whereas translating both means to $m$ changes it by $-3m^2$. This stationary point has both improving and worsening directions. Identical initialization cannot discover the separation because every observation keeps identical label probabilities. Trying several distinct initial fits and comparing their final log-likelihoods is useful, but does not prove global optimality.

Keeping the variance fixed also removes a different pathology. With known $\sigma =1$, every component probability density function is at most $(2\pi {)}^{-1/2}$, so $\ell \le -3\log (2\pi )$. If separate component variances are allowed to shrink without restriction, one mixture component can concentrate at an observation while another retains positive density at the other observations, making the likelihood function unbounded. The fixed positive variance in this example prevents that collapse; it does not remove initialization traps.