= EM expectation step
{c}
{title2=$Q(\theta\mid\theta^{(t)})$}
= E-step
{c}
{synonym}
Hold the old <statistical parameters> fixed when taking the <conditional expectation>, while keeping the candidate <statistical parameters> $\theta$ variable:
$$
q_t(z)=p_{\theta^{(t)}}(z\mid y),\qquad Q(\theta\mid\theta^{(t)})=\mathbb E_{q_t}[\log p_\theta(y,Z)].
$$
For <independent> observations in a <finite mixture model>, $Z_i$ is a hidden component label. <Bayes' theorem> gives its <mixture responsibilities>:
$$
r_{ij}^{(t)}=\mathbb P_{\theta^{(t)}}(Z_i=j\mid Y_i=y_i)=\frac{\pi_j^{(t)}f_j(y_i;\theta^{(t)})}{\sum_k\pi_k^{(t)}f_k(y_i;\theta^{(t)})},\qquad \sum_jr_{ij}^{(t)}=1.
$$
These <posterior probabilities> are the <conditional expectations> of the <mixture allocation indicators>; they turn hidden counts into fractional counts.<Maximum Likelihood from Incomplete Data via the EM Algorithm by Arthur Pentland Dempster and others>[Section 4.3, printed pp. 15-16, especially the paragraph following equation (4.3.3): conditional indicator probabilities and weighted counts]{ref} They condition on a current fixed fit, rather than averaging over a <posterior distribution> of the <statistical parameters>. Taking the <conditional expectation> of the complete-data <log-likelihood> is essential: substituting a single guessed label, or merely substituting the <conditional mean> into a nonlinear complete-data expression, generally gives a different algorithm.
Back to article page
