Generalized linear mixed model

A generalized linear mixed model adds latent random effects to the linear predictor of a generalized linear model.

A subject has a random intercept $B_i\sim N(0,{\tau }^2)$, with responses conditionally independent given $B_i$ and their covariates. Integrating the product of Bernoulli likelihoods over $B_i$ defines the joint subject likelihood and induces within-subject dependence. Its coefficients describe conditional odds ratios for a fixed latent subject effect. Under missing at random with distinct parameters, the observed-response likelihood integrates over the same random effect and sums out unobserved responses. Dependence of dropout on $B_i$ is not generally ignorable merely because conditional independence holds given $B_i$.

Put $p_B={\mathrm{logit}}^{-1}(\eta +B)$ and $M(\eta )=\mathbb{E}(p_B)$. Differentiation gives $M^{\prime }=\mathbb{E}[p_B(1-p_B)]=M(1-M)-\mathrm{Var}(p_B)$. Hence the displayed derivative lies between zero and one, strictly below one for a nondegenerate random intercept. A marginal logit is therefore generally nonlinear and its slope differs from the conditional slope even without confounding. This is a precise instance of noncollapsibility of the odds ratio.

A Gaussian linear mixed model combines fixed effects, random effects with a normal distribution, and independent Gaussian noise. For $b\sim N(0,D)$ and $\epsilon \sim N(0,R)$ independently, its marginal distribution is $Y\sim N(X\beta ,V)$ with $V=ZD{Z}^T+R$. The random effects induce dependence among observations sharing their design columns. A random-intercept linear mixed model and a random-slope linear mixed model are special cases.

A subject-specific random intercept and random slope may have a joint bivariate normal distribution with arbitrary positive-definite covariance matrix $D$. With independent normal measurement errors of variance ${\sigma }^2$, the marginal covariance between a person's readings at $s,t$ is $D_{00}+(s+t)D_{01}+st{D}_{11}$, plus ${\sigma }^2$ on the diagonal. Different subjects remain independent. Allowing $D_{01}$ distinguishes this model from an independent random-intercept and random-slope model.

A variance component is a nonnegative parameter multiplying a specified covariance contribution. In a Gaussian linear mixed model with independent standardized group effects, ${\tau }_k^2{Z}_k{Z}_k^T$ describes the covariance induced by one set of random effects. Testing whether a component is zero is a variance-component likelihood-ratio test at a boundary; ordinary regular chi-squared likelihood-ratio calibration need not apply.

A method-of-moments variance component estimate equates residual mean squares in ANOVA to their model expectations and solves for the variance components. An unconstrained estimate can be negative even when the model component must be nonnegative; boundary treatment then requires a stated estimation convention. The method gives point estimates, not certainty about future experiments.

With known covariance parameters, best linear unbiased prediction minimizes prediction-error variance over linear predictors that are unbiased over both observation errors and random effects, for every fixed-effect value. In a Gaussian linear mixed model, the random-effect predictor is $D{Z}^T{V}^{-1}(Y-X{\hat{\beta }}_{\mathrm{GLS}})$, where generalized least squares estimates the fixed effects. Plugging in covariance estimates gives an empirical predictor; its uncertainty must also account for estimating those covariance parameters. It differs from a best linear unbiased estimator of an unknown fixed coefficient.

In a Gaussian linear mixed model with $V=ZD{Z}^T+R$, the conditional multivariate normal distribution of $b$ given $Y$ has mean $D{Z}^T{V}^{-1}(Y-X\beta )$ and covariance matrix $D-D{Z}^T{V}^{-1}ZD$. When nonsingular, its mean is also its mode. Estimated parameters yield empirical conditional modes and shrink group deviations toward zero. At a zero variance component the associated effect is degenerate at zero; the covariance formula still applies, whereas formulas involving $D^{-1}$ require limits.

For $\kappa >0$, this error correlation decreases exponentially with elapsed time and permits irregular observation times. It is the stationary correlation of an Ornstein-Uhlenbeck process. Within a Gaussian linear mixed model, apply it to the errors conditional on random effects; the marginal correlation also includes those effects. nlme::corCAR1 parametrizes the same correlation by $\rho =e^{-\kappa }\in (0,1)$ and permits separate grouped time series.

Independent Gaussian random intercepts $u_j$ and random slopes $v_j$ give marginal within-group covariance ${\tau }_0^2+{\tau }_1^2{t}_{ij}{t}_{kj}+{\sigma }^2{1}_{\{i=k\}}$. In lme4, separate terms (1 | group) and (0 + x | group) impose zero intercept-slope covariance; (1 + x | group) estimates it. This independence restriction depends on the predictor origin: replacing $t$ by $t-c$ transforms the intercept effect to $u_j+c{v}_j$, generally correlated with $v_j$.

A Poisson generalized linear mixed model gives a count response a conditional Poisson distribution whose logarithmic mean contains both fixed effects and latent random effects.

Let the positive multiplier $U_i$ have a gamma distribution with mean $\tau$ and variance $\theta$, and conditionally let the repeated counts be independent with means $U_i{a}_{ij}$. Total expectation and variance giveThe Poisson-gamma mixture gives negative-binomial marginal counts of size ${\tau }^2/\theta$. Despite the shared multiplier, the marginal correlations generally depend on both marginal means and need not be exchangeable.

When $a_{ij}=\exp ({\beta }_0+{\beta }^{\mathsf{T}}{x}_{ij})$, replacing $U_i$ by $c{U}_i$ and the fixed intercept by ${\beta }_0-\log c$ leaves every conditional count mean and the integrated data law unchanged. Thus without fixing the random multiplier's mean, the gamma random-intercept Poisson model identifies the marginal intercept ${\beta }_0+\log \tau$ and relative heterogeneity $\theta /{\tau }^2$, rather than these three parameters separately. Normalizing $\tau =1$ removes this scale ambiguity.

Given Gaussian random intercept $b_0$ and random slope $b_1$, let the conditional incidence rate be $e^{{\beta }_0+{\beta }_{1}s+b_0+b_{1}s}$. The normal moment-generating function gives the displayed marginal rate, with $\mathrm{Var}(b_0+b_{1}s)={\tau }_0^2+2s{\tau }_{01}+s^2{\tau }_1^2$. Thus the conditional annual rate ratio for a cluster is $e^{{\beta }_1+b_1}$, whereas the marginal rate ratio also contains covariance and variance terms. Centering $s$ at a chosen year makes intercept heterogeneity refer directly to that year.

A random intercept gives every group a latent additive shift, inducing dependence among observations from the same group.

For a conditional count mean $\mathbb{E}(Y\mid x,b)=\exp (b+{\beta }_0+{\beta }^{\mathsf{T}}x)$, suppose the distribution of $b$ is independent of the covariates and has finite $\mathbb{E}{e}^b$. Integrating over $b$ gives the displayed logarithmic link function mean: every slope stays unchanged, and only the intercept shifts. This is an exact property of the log link; it does not hold in this form for the logit.

A random-intercept linear mixed model has the formwhere independent group effects $b_i$ induce positive covariance among observations from the same group.