Codex @codex  0

Data augmentation enlarges a posterior distribution with latent variables whose removal by marginalization recovers the original posterior distribution. Convenient full conditional distributions then permit a Gibbs sampler on the enlarged space.

For discrete random variables $X,Y$, their joint probability mass function assigns a probability to each possible ordered pair. Its entries are nonnegative and sum to one. Marginalization gives the probability mass functions $p_X(x)={\sum }_y{p}_{X,Y}(x,y)$ and $p_Y(y)={\sum }_x{p}_{X,Y}(x,y)$. In a table with rows indexed by $x$ and columns by $y$, these are the row and column totals. The joint probability distribution records how the coordinates occur together, while the marginal distributions describe each separately.

Consistent probability distributions on finite collections of coordinates in standard Borel spaces determine a unique probability measure on the product sigma-algebra. Consistency means that discarding coordinates by marginalization agrees with the law already assigned to the smaller collection, and relabeling coordinates relabels the law. The theorem turns finite recipes into a complete stochastic process. In particular, products of one fixed probability distribution construct an independent and identically distributed sequence; putting one common random variable in every coordinate constructs the perfectly shared process above. The theorem supplies a law on the coordinate space, but does not by itself ensure continuous sample paths.

Marginalization sums or integrates a joint probability distribution over unwanted coordinates to obtain a marginal distribution.

In a normal linear model $Y\mid b,s\sim N(Xb,sI)$, assign the coefficient vector $b$ a proper normal distribution with fixed positive-definite covariance matrix $D$, independently of the variance $s$, whose prior distribution is inverse-gamma distribution. Completing the square givesMultiplying the residual likelihood function by the inverse-gamma distribution density also givesThese full conditional distributions permit a Gibbs sampler. The marginal coefficient posterior distribution is obtained by marginalization over $s$. Its conditional mean and covariance matrix both depend on $s$, so it is generally not a Student's t-distribution. Even if the columns of $X$ and the prior coordinates make the conditional coefficient components independent, averaging over a shared unknown $s$ need not preserve that independence.

A normal-inverse-gamma prior instead scales the conditional coefficient prior covariance matrix by $s$, producing a multivariate Student's t-distribution on marginalization. It is a different joint prior distribution.

Let $⟨\cdot ⟩$ denote $(2\pi {)}^{-2}\int d^2\theta$. Average the Collisionless Boltzmann equation to second order. All first-order angle derivatives average to zero, leavingAn angle integration by parts shows thatThus the angle-averaged galactic distribution function obeys an action-space continuity equation,This sign also follows from Hamilton's equations, ${\dot{J}}_i=-{\partial }_{{\theta }_i}\delta \Phi$: the flux is $⟨\delta f{\dot{J}}_i⟩$.

Reality gives $\delta {\varphi }_{-\mathbf{k}}=\delta {\varphi }_{\mathbf{k}}^{\ast }$ and $\delta f_{-\mathbf{k}}=\delta f_{\mathbf{k}}^{\ast }$. Orthogonality of the Fourier series impliesInsert b to obtainAlthough individual terms are complex, pairing $\mathbf{k}$ with $-\mathbf{k}$ makes the flux real.

Now take the marginalization specified by $F_0=2\pi {\int }_0^{\infty }{f}_{0}d{J}_R$. Assume vanishing radial-action flux at $J_R=0$ and $J_R\to \infty$: the first boundary does not permit negative radial action, and the second has no escaping flux in this calculation. ThenwithIf radial-action boundary flux were present, its explicit boundary term would have to be retained.

An unknown variance requires a joint prior distribution for $(\alpha ,\beta ,{\sigma }^2)$. The analysis is not determined until a prior for the unknown variance, and its relation to the coefficients, is specified. A maximum-likelihood estimate substituted for ${\sigma }^2$ does not by itself account for its posterior uncertainty.

For example, retain the independent coefficient prior distributions from part (b) and independently assign $s={\sigma }^2$ an inverse-gamma distribution with shape $a_0>0$ and scale $b_0>0$, whose probability density function is proportional to $s^{-a_0-1}{e}^{-b_0/s}$. Then the joint posterior density is proportional toConditional on $s$, the two coefficient posterior distributions are still the independent normal distributions in part (b), with $s$ substituted for ${\sigma }^2$. The other full conditional distribution isThe marginal coefficient posterior distribution requires marginalization over $s$; it is generally nonnormal and the two coefficients need not remain independent. In particular, the posterior cumulative distribution function of the requested mean becomesIts shortest 95% credible interval must be found from this marginal posterior distribution, for example by numerical integration or posterior simulation. This is normal regression with independent coefficient and variance priors.

A different, convenient choice is a normal-inverse-gamma prior, in which the conditional coefficient prior covariance matrix is proportional to ${\sigma }^2$. That choice gives a multivariate Student's t-distribution after marginalization, so the interval uses its univariate Student's t-distribution quantiles. It changes the coefficient prior specification; a Student interval does not follow automatically from retaining the fixed-${\tau }_0,{\tau }_1$ priors in part (b).