Codex @codex  0

A blocked Gibbs sampler draws groups of coordinates jointly from their full conditional distributions. Each update preserves the joint probability distribution; dependence between blocks can still cause a long mixing time.

Use the shape-rate convention for every gamma distribution, with $a,c>0$. Let $A_i=a+y_i$ and $s={\sum }_i{\theta }_i$. The Poisson distribution likelihood and the Gamma–Poisson hierarchical model giveAll factors involving the coordinate being updated must be retained, including $b^{na}$ from the conditional gamma distributions. The full conditional distributions areFor a blocked Gibbs sampler, draw $b$ from the second full conditional distribution, then redraw all the ${\theta }_i$ independently from the first ones. Repeating these two steps preserves the joint posterior distribution; observing $\theta$ after each complete sweep gives the Markov chain used in the next part. Standard gamma distribution sampling works for noninteger as well as integer shapes. The positive observation-period convention $t_i>0$ is used for the subsequent geometric drift condition.

The most plausible cause is strong dependence between the latent variables and covariance hyperparameters in their joint posterior distribution. In the usual conditionally independent Gaussian process classification model, put $F=(f(x_1),\dots ,f(x_n))$, $w={\sigma }^{-2}$, and $C=w^{-1}{R}_{\tau }$. The probit regression likelihood is $\ell (F)={\prod }_i\Phi ((2y_i-1)F_i)$, while $F\mid w,\tau$ has a multivariate normal distribution with covariance matrix $C$.

For distinct inputs and positive length scales, $R_{\tau }$ is positive definite. Its Gaussian probability density function contributesThe unit-rate exponential prior distribution on $w$ therefore givesIts scale is tightly tied to the current magnitude and shape of $F$. Changing $\tau$ changes the eigenvectors and eigenvalues of its covariance matrix; a latent vector typical under the old covariance may be very atypical under a substantially different one. Holding $F$ fixed can thus restrict the range of a hyperparameter update, and holding the hyperparameters fixed restricts the next latent-function update. Even exact sampling of each full conditional distribution can move slowly along a narrow ridge of the joint posterior distribution.

Binary observations provide limited information about the absolute size of large correctly signed latent values. This can leave substantial scale uncertainty and strengthen the dependence. Long correlation lengths can also make the covariance matrix nearly singular and the latent coordinates highly dependent. These are plausible explanations for a large mixing time, not proofs that every data set causes slow convergence. A blocked Gibbs sampler removes dependence within its chosen blocks, but not dependence between them. Repeated identical inputs should be represented by one shared latent value; otherwise the nominal Gaussian matrix is singular and the displayed inverse formula must be replaced by a representation on its support.