= Hyperprior
{title2=$p(\lambda)$}
A \b[hyperprior] is a <prior distribution> assigned to a <hyperparameter> that controls another <prior distribution>. In a <hierarchical Bayesian model>, this makes uncertainty about the prior's parameters part of the joint probability model, rather than replacing those parameters by fixed numbers.<Stan User's Guide by Stan Development Team>[Version2.37, Regression Models, Hierarchical priors]{ref}
For data $y$, a model parameter $\theta$ and a <hyperparameter> $\lambda$, a common factorization is
$$
p(y,\theta,\lambda)
=p(y\mid\theta)\,p(\theta\mid\lambda)\,p(\lambda).
$$
Here $p(y\mid\theta)$ supplies the <likelihood function>, $p(\theta\mid\lambda)$ is the conditional <prior distribution>, and $p(\lambda)$ is the <hyperprior>. After observing $y$, <Bayesian inference> gives a joint <posterior distribution>. The marginal posterior of $\theta$ averages over the updated hyperparameter distribution:
$$
p(\theta\mid y)=\int
p(\theta\mid\lambda,y)\,p(\lambda\mid y)\,d\lambda.
$$
For discrete hyperparameters, the integral is replaced by a sum.
An exact example makes the averaging visible. Let $\theta$ be a success probability and choose a two-point <hyperprior>,
$$
\Pr(\kappa=2)=\Pr(\kappa=20)=\frac12,\qquad
\theta\mid\kappa\sim
\operatorname{Beta}(\kappa/2,\kappa/2).
$$
Both conditional <beta distributions> have <expectation> $1/2$, but they concentrate differently around it. Now observe two successes that are independent conditional on $\theta$, each with a <Bernoulli distribution>. Their conditional probability is $\theta^2$. Integrating against the two possible priors gives
$$
p(y\mid\kappa=2)=\frac13,\qquad
p(y\mid\kappa=20)=\frac{11}{42}.
$$
Thus the data also update the <hyperparameter>:
$$
\Pr(\kappa=2\mid y)=\frac{14}{25},
\qquad
\Pr(\kappa=20\mid y)=\frac{11}{25}.
$$
The conditional <posterior distributions> of $\theta$ are respectively $\operatorname{Beta}(3,1)$ and $\operatorname{Beta}(12,10)$. Their posterior mixture has mean
$$
\mathbb E[\theta\mid y]
=\frac{14}{25}\frac34+\frac{11}{25}\frac{12}{22}
=0.66.
$$
This is an original illustrative probability model, rather than data or a numerical example from the guide.
Fixing $\kappa$ at its prior mean $11$ would instead produce $\operatorname{Beta}(7.5,5.5)$ after the same observations, with posterior mean $7.5/13\approx0.577$. Averaging the hyperparameter first has changed the model; it does not reproduce averaging the conditional posteriors with their updated probabilities. A <hyperprior> therefore adds a specified layer of uncertainty, while its own choice remains part of the assumptions of the <hierarchical Bayesian model>.
Back to article page
