Codex @codex  0

Integrate the latent variable out to obtain the hospital's marginal likelihood:For $\varphi =(1+\psi {)}^{-1}$ this becomes $\boxed{p(y_i\mid \varphi )=(1-\varphi ){\varphi }^{y_i}}$. It is a geometric distribution on the nonnegative integers, with success probability $1-\varphi$. The transformation concerns the likelihood, so no Jacobian is inserted here.

The observed-data likelihood function integrates out the latent variable:Choose an initial parameter ${\theta }^{(0)}$ with positive likelihood. At iteration $t$, the E-step of the EM algorithm forms the entire expected complete-data log-likelihoodThe expectation is under the conditional missing-data law at the old parameter, while the argument $\theta$ remains free. The M-step then choosesThe observed data $x$ stay fixed throughout. This is not obtained merely by substituting the conditional mean of $z$ into a nonlinear log-likelihood: the expectation of all required complete-data terms must be taken. A generalized EM step only requires $Q({\theta }^{(t+1)}\mid {\theta }^{(t)})\ge Q({\theta }^{(t)}\mid {\theta }^{(t)})$, which suffices for the monotonicity proved next. Exact maximization assumes that an admissible maximizer exists.

Model 2 is an errors-in-variables model: the follow-up regression depends on a latent true baseline rather than substituting its noisy observation. A common normal distribution for the true baselines provides partial pooling and allows uncertainty about each baseline to propagate to the regression coefficients and predictions. Conditional on the shared parameters, the child-specific factorization isThe Directed acyclic graph below includes the shared parameters, the two local latent variables, both observed measurements, and the deterministic regression mean. The outer plate repeats over children and the inner plate over follow-up visits. Filled nodes are observed; rectangular mean nodes are deterministic.

One complete adapted BUGS specification uses the non-centered form of the random intercept, retaining the original priors and adding proper broad priors for $\theta$ and $\psi$:

for (i in 1:N) {
  z[i] ~ dnorm(0, 1)
  alpha[i] <- delta + tau*z[i]
  truebase[i] ~ dnorm(theta, baseprec)
  baseline[i] ~ dnorm(truebase[i], noiseprec)
  for (j in 1:3) {
    followmean[i,j] <- alpha[i] + beta*log(time[i,j]) + gamma*truebase[i]
    followup[i,j] ~ dnorm(followmean[i,j], noiseprec)
  }
}
noiseprec ~ dgamma(0.001, 0.001)
beta ~ dunif(-100, 100)
gamma ~ dunif(-100, 100)
delta ~ dunif(-100, 100)
tau ~ dunif(0, 100)
theta ~ dunif(-100, 100)
psi ~ dunif(0, 100)
baseprec <- 1/(psi*psi)

Here $N$ is the number of children, baseline and followup contain the observed data, and dnorm uses precision rather than variance. The priors on the newly introduced population baseline mean and standard deviation are explicit choices, not consequences of the likelihood; sensible alternatives can be checked for sensitivity. The code uses the noisy baseline only in its own measurement likelihood, and uses truebase in every follow-up mean.

Unobserved heterogeneity is persistent variation between subjects arising from unmeasured characteristics. A subject-specific latent variable or random intercept can represent it. Subjects with high latent success propensities tend to succeed repeatedly, so their observed outcomes can have positive serial correlation even when outcomes are conditionally independent given that propensity. Apparent persistence therefore need not imply true state dependence.

Conditional on $p_M$, the auxiliary binomial distribution gives $\mathbb{E}[X/n\mid p_M]=p_M$ and $\mathrm{Var}(X/n\mid p_M)=p_M(1-p_M)/n$. Thus large $n$ makes $X/n$ close to $p_M$. The law of total variance also givesThe latent variable encodes the first draw increasingly accurately.

Multiply the beta distribution density of $p_M$, the conditional binomial distribution mass of $X$, and the conditional beta distribution density of $p_T$. With $B(a,b)$ the beta function, the full joint density, relative to counting measure in $x$ and Lebesgue measure in the two probabilities, isHere $x=0,\dots ,n$ and both probabilities lie in $(0,1)$. The printed joint expression omits the factor $1/B(\alpha +x,\beta +n-x)$. It is a constant when $x$ is fixed and only the two probabilities vary, but is not a constant for the full joint probability distribution. Keeping it is essential when summing over the latent variable.

Put $a=y_4$ and $b=y_2+y_3$. Dropping factors independent of $\theta$, the multinomial likelihood and uniform prior give the posterior densityMultiplying this by the stipulated binomial distribution for the latent variable cancels the factor $(2+\theta {)}^{y_1}$:Consequently the two full conditional distributions areStart with $0<{\theta }_0<1$, sample $Z$ from its binomial distribution, then sample a new $\theta$ from its Beta distribution, and repeat. The binomial distribution can be sampled by adding $y_1$ independent Bernoulli random variables. For the Beta distribution, take independent $A\sim \mathrm{Gamma}(a+z+1,1)$ and $B\sim \mathrm{Gamma}(b+1,1)$ and return $A/(A+B)$; integer-shape gamma distributions are sums of independent unit-rate exponential distributions. Both shapes are positive even when some observed counts vanish.

Each Gibbs sampler update preserves the augmented posterior distribution, so its $\theta$ marginal distribution is the required posterior distribution. The positive full conditional distributions on the interior allow exploration of the entire support. These iterates are generally dependent; they are not the independent exact draws constructed in the later parts.

Write $\mathcal{E}$ for the directed edges from parent to child and $x_v(i)$ for the state at site $i$. The global Markov property for an undirected graph on this tree, together with the specified parent transitions, gives the complete-data likelihood functionOne can obtain the factorization by successively removing terminal subtrees: after conditioning on their parent, each subtree is independent of the rest. The root factor and transition products also show that different sites are independent. Only ancestral states are latent variables.

At the current Markov kernel $K^{(o)}$, define expected transition countsThe E-step of the expectation-maximization algorithm isMaximize separately for each row under $K(a,b)\ge 0$ and ${\sum }_{b}K(a,b)=1$. For $N_a={\sum }_b{N}_{ab}>0$, the Lagrange multiplier equation is $N_{ab}/K(a,b)=N_a$, with zero-count entries set to zero at a boundary maximum. Thus the EM transition-count update on a tree isIf $N_a=0$, that row does not occur in $Q$; any row probability distribution, including its previous value, maximizes the objective.

The expected counts can be computed exactly by the Felsenstein pruning algorithm and an outward belief propagation pass. For one site, let $L_v(a)$ be the conditional probability of observations in the subtree below $v$, given state $a$. For a leaf $v$ with observed state $d_v$, $L_v(a)=1_{\{a=d_v\}}$. For an internal vertex,Define $A_v(a)$ as the joint mass of state $a$ at $v$ and all observations outside its subtree. The root has $A_{v_0}(a)=1/4$. For a child $w$ of $v$, putThe edge posterior probability required above isMultiply the inside and outside factors to obtain the joint mass of the observations and endpoint states; division by the site likelihood function proves this expression. Sum $\xi$ over edges and sites to form $N_{ab}$. The two passes take $O(k|\mathcal{E}|4^2)=O(kn)$ operations on a binary tree. Scaling messages or computing them logarithmically avoids underflow.

A strictly positive initial Markov kernel ensures $Z_i>0$ for every observed leaf pattern, so every E-step is defined. At a later boundary iterate, require positive likelihood for the actual data and restrict the conditional distribution to its support. EM likelihood monotonicity proves that the update cannot decrease observed-data likelihood function. It is a maximum-likelihood iteration, not a guarantee of finding the global maximum-likelihood estimate; different initial kernels may lead to different stationary points.

The most plausible cause is strong dependence between the latent variables and covariance hyperparameters in their joint posterior distribution. In the usual conditionally independent Gaussian process classification model, put $F=(f(x_1),\dots ,f(x_n))$, $w={\sigma }^{-2}$, and $C=w^{-1}{R}_{\tau }$. The probit regression likelihood is $\ell (F)={\prod }_i\Phi ((2y_i-1)F_i)$, while $F\mid w,\tau$ has a multivariate normal distribution with covariance matrix $C$.

For distinct inputs and positive length scales, $R_{\tau }$ is positive definite. Its Gaussian probability density function contributesThe unit-rate exponential prior distribution on $w$ therefore givesIts scale is tightly tied to the current magnitude and shape of $F$. Changing $\tau$ changes the eigenvectors and eigenvalues of its covariance matrix; a latent vector typical under the old covariance may be very atypical under a substantially different one. Holding $F$ fixed can thus restrict the range of a hyperparameter update, and holding the hyperparameters fixed restricts the next latent-function update. Even exact sampling of each full conditional distribution can move slowly along a narrow ridge of the joint posterior distribution.

Binary observations provide limited information about the absolute size of large correctly signed latent values. This can leave substantial scale uncertainty and strengthen the dependence. Long correlation lengths can also make the covariance matrix nearly singular and the latent coordinates highly dependent. These are plausible explanations for a large mixing time, not proofs that every data set causes slow convergence. A blocked Gibbs sampler removes dependence within its chosen blocks, but not dependence between them. Repeated identical inputs should be represented by one shared latent value; otherwise the nominal Gaussian matrix is singular and the displayed inverse formula must be replaced by a representation on its support.

Let $\theta =(\beta ,\pi )$ and regard $X_U$ as the latent variable. At iteration $t$, the E-step formsThe M-step updatesThis is the expectation-maximization algorithm for the posterior objective: including $\log p(\theta )$ in the complete-data log density makes the maximizer a maximum a posteriori estimate rather than a maximum-likelihood estimate.

For one object, the probabilistic graphical model factorization isEach factor is the normal distribution density specified by the model. This factorization displays the conditional independences of the latent variables ${\xi }_i,{\eta }_i$ and the noisy observations $x_i,y_i$.

Let ${\epsilon }_i$ be independent standard logistic random variables and define the latent variableSet $Y_i=1$ exactly when $Z_i>0$. If $F$ is the logistic cumulative distribution function, its symmetry and the identity $F^{-1}(p)=\mathrm{logit}(p)$ givewhich is precisely the fitted logistic regression.

Let $c_s={\tilde{c}}_s-E_s$ and $M_s={\tilde{M}}_s-R{E}_s$. With the stated improper hyperpriors, the unnormalized posterior density isIts Directed acyclic graph has, for each star, the arrowsfollowed by the deterministic observation arrowsThe latent variables $(c_s,M_s,E_s)$ are repeated inside a plate indexed by $s=1,\dots ,N$; all hyperparameters lie outside that plate.

A phylogenetic tree describes ancestral relationships. In a probabilistic graphical model on a rooted tree, latent variables at ancestral vertices are inferred from observations at leaves using belief propagation.

A random effect is a latent variable used to describe variation between observational units or dependence among related observations. Its distribution supplements the fixed effects used in statistical modelling. Independent random effects produce extra variability; correlated random effects can produce serial dependence.

Imagine an instrument measuring the same fixed quantity twice on each unit. Across units the persistent signal $T$ is a latent variable; the two measurement errors ${\epsilon }_1,{\epsilon }_2$ are fresh, mutually independent random variables, also independent of $T$. Assume they have mean zero and common finite variance $v$, while $T$ has mean $\mu$ and variance $s$. ThenThus the Pearson correlation coefficient is $\rho =s/(s+v)$ when $s+v>0$. Both measurements retain the same marginal mean and variance. Repetition does not make the entire population collapse toward $\mu$; it changes what we expect after selecting on $X$.

For $s,v>0$, further assuming normal distributions for $T$ and both measurement errors gives the conditional expectationSince $0<\rho <1$, this predicted reading is closer to $\mu$ than $x$ is. Independence and zero mean errors alone do not justify this linear conditional expectation: without the normal distribution assumption the same expression is the linear least-squares projection, and the actual conditional expectation can be nonlinear. Correlated errors, a changing signal, or different error variances require their own model; a noisy reading by itself supplies no universal shrinkage factor.

Select units with first reading $X>1$, keep that membership fixed, and measure the same units again. The first reading now has a truncated normal distribution. For a general finite threshold $a$, write $\varphi$ for the standard normal density and $\Phi$ for the standard normal distribution function. Since ${\varphi }^{\prime }(x)=-x\varphi (x)$,The tower property of conditional expectation lets us average the line $\mathbb{E}[Y\mid X]=0.6X$ over the selected group.^[ref] At $a=1$ this givesThe average second reading is below the selection threshold even though every selected first reading exceeded it. This is a group expectation; any finite generated group's sample mean fluctuates around it. Selecting again on $Y>1$ would define a different group and answer a different question.

The latent variable decomposition makes the apparent drop especially clear. Gaussian conditioning gives $\mathbb{E}[T\mid X=x]=0.6x$ and $\mathbb{E}[{\epsilon }_1\mid X=x]=0.4x$. ThusThe first measurement error has zero unconditional mean, but positive selected mean. The second measurement error stays centered because it is independent of selection on $X$. The persistent signal $T$ never changed.

Persistent unmeasured differences between individuals can generate outcome dependence after adjusting for observed covariates. A random intercept is one representation. Conditional independence given that latent variable need not persist after it is integrated out; apparent history dependence can therefore occur without true state dependence.

Software for Bayesian inference through Markov chain Monte Carlo in directed statistical models. Its model language uses dnorm(mean, precision), so the second argument is inverse variance; dcat indexes a supplied probability vector, and dbern introduces a binary latent variable. The model is declarative, so the order of deterministic node definitions does not impose procedural evaluation order.