= Normal-gamma posterior with a flat prior
{title2=$\pi(u,\gamma\mid m)$}
Let $m\mid u,\gamma\sim\mathcal N(Au,\gamma^{-1}I_k)$, let the <prior distribution> of the precision be the shape-rate <Gamma distribution> $\operatorname{Gamma}(\alpha,\beta)$, and use a constant <improper prior> for $u\in\mathbb R^d$. Assume $A$ has zero <null space>. Write $B=A^TA$, $u_*=B^{-1}A^Tm$, $a=\alpha+k/2$, $b_0=\beta+\|m-Au_*\|^2/2$, and $s=\alpha+(k-d)/2>0$. The proper joint <posterior density> is proportional to
$$
\gamma^{a-1}\exp\left[-\gamma\left(b_0+\tfrac12(u-u_*)^TB(u-u_*)\right)\right].
$$
Its <conditional distributions> are $u\mid\gamma,m\sim\mathcal N(u_*,\gamma^{-1}B^{-1})$ and $\gamma\mid u,m\sim\operatorname{Gamma}(a,\beta+\|m-Au\|^2/2)$. Its marginal precision law is $\operatorname{Gamma}(s,b_0)$. Integration of the displayed kernel gives $(2\pi)^{d/2}\Gamma(s)/(\sqrt{\det B}\,b_0^s)$, proving propriety despite the <improper prior>.
A joint <maximum a posteriori estimate> relative to $du\,d\gamma$ exists exactly when $a>1$, and is $(u_*,(a-1)/b_0)$. If $a=1$, the supremum is approached at $\gamma=0$ but not attained; if $a<1$, the density is unbounded there. Separate marginal maximization gives $u_*$ for the unknown and $(s-1)/b_0$ for the precision when $s>1$. This illustrates that joint and marginal modes need not agree, and that a proper <posterior distribution> need not have a mode in its open parameter space.
Back to article page
