Normal regression with independent coefficient and variance priors
ID: normal-regression-with-independent-coefficient-and-variance-priors
In a normal linear model , assign the coefficient vector a proper normal distribution with fixed positive-definite covariance matrix , independently of the variance , whose prior distribution is inverse-gamma distribution. Completing the square givesMultiplying the residual likelihood function by the inverse-gamma distribution density also givesThese full conditional distributions permit a Gibbs sampler. The marginal coefficient posterior distribution is obtained by marginalization over . Its conditional mean and covariance matrix both depend on , so it is generally not a Student's t-distribution. Even if the columns of and the prior coordinates make the conditional coefficient components independent, averaging over a shared unknown need not preserve that independence.
A normal-inverse-gamma prior instead scales the conditional coefficient prior covariance matrix by , producing a multivariate Student's t-distribution on marginalization. It is a different joint prior distribution.
New to topics? Read the docs here!