= Gaussian likelihood Gibbs annealing
{c}
{title2=$(\mu_n,v_n)\longrightarrow(\bar x,S/m)$}
Let $\bar x=m^{-1}\sum_i x_i$ and $S=\sum_i(x_i-\bar x)^2>0$. With Lebesgue base measure $d\mu\,dv$ for $v=\sigma^2$, the likelihood-power law has conditionals
$$
\mu\mid v\sim N(\bar x,Tv/m),\qquad
v\mid\mu\sim\operatorname{InvGamma}\left(\frac{m}{2T}-1,\frac{S+m(\mu-\bar x)^2}{2T}\right).
$$
The joint law is proper for $T<m/3$, since its marginal $v$ is <inverse-gamma distribution> with shape $m/(2T)-3/2$ and scale $S/(2T)$. There is also a direct convergence proof for successive <Gibbs sampling> updates with deterministic $0<T_n\leq m/10$ and $T_n\to0$. Update $\mu_n$ from $v_{n-1}$, then $v_n$ from $\mu_n$. <Inverse-gamma distribution> moments and conditional normal moments give
$$
\mathbb E v_n=\frac{S+T_n\mathbb E v_{n-1}}{m-4T_n},\qquad
\mathbb E v_n^2=\frac{S^2+2ST_n\mathbb E v_{n-1}+3T_n^2\mathbb E v_{n-1}^2}{(m-4T_n)(m-6T_n)}.
$$
The coefficients of the preceding moments are bounded by $1/6$ and $1/8$, so these moments stay bounded from a finite deterministic initial <variance>. Letting $T_n\to0$ gives $\mathbb E v_n\to S/m$ and $\mathbb E v_n^2\to(S/m)^2$. Also $\mathbb E(\mu_n-\bar x)^2=T_n\mathbb E v_{n-1}/m\to0$. Thus the iterates converge in <mean> square to the normal <maximum-likelihood estimate>.
Back to article page
