Finite Gaussian mixture with a common variance ($f(y)={\sum }_j{\pi }_j\varphi (y;{\mu }_j,{\sigma }^2)$)

A finite Gaussian mixture with a common variance has density $f(y)={\sum }_{j=1}^k{\pi }_j\varphi (y;{\mu }_j,{\sigma }^2)$, with positive common variance and weights summing to one. The means locate its latent components, which need not correspond to distinct modes. It has $2k$ free parameters: $k-1$ weights, $k$ means and one variance. EM for Gaussian mixtures with a common variance supplies iterative likelihood updates.

A mixture regression with shared slopes models $Y_i\mid Z_i=j,x_i\sim N({\alpha }_j+x_i^T\beta ,{\sigma }^2)$ with common slopes and component-specific intercepts. It jointly estimates mean adjustment and latent groups, rather than fitting a mixture to fixed residuals. If both a global intercept and component offsets are included, an identifying constraint is required. Constant weights assume latent membership proportions do not depend on the covariates.

Residual mixture clustering first removes a fitted covariate-dependent mean and then explores latent groups in the regression residuals. It seeks differences relative to the adjustment rather than raw-response differences. Mixture responsibilities express uncertain membership. The fitted mixture alone does not establish distinct real-world populations, and two-stage residual mixture fitting ignores some adjustment uncertainty.

Fitting a mixture model to regression residuals treats the fitted mean adjustment as fixed. Even with independent homoscedastic errors, ordinary least-squares residuals have covariance matrix ${\sigma }^2(I-H)$ and satisfy fitted linear constraints, so they are not exactly independent observations. A joint mixture regression with shared slopes or a bootstrap of the whole fitting process better accounts for the first-stage estimation uncertainty.

The E-step computes mixture responsibilities ${\tau }_{ij}$ from the old parameters. Put $N_j={\sum }_i{\tau }_{ij}$. The M-step updates ${\pi }_j=N_j/n$, ${\mu }_j={\sum }_i{\tau }_{ij}{y}_i/N_j$ and ${\sigma }^2={\sum }_{i,j}{\tau }_{ij}(y_i-{\mu }_j^{\mathrm{new}}{)}^2/n$. The variance uses new means and old responsibilities. EM likelihood monotonicity guarantees nondecrease of the observed likelihood, not a global optimum.