Generalized likelihood-ratio test

A generalized likelihood-ratio test compares the likelihood maximized under the null with the likelihood maximized over the full parameter space.

Let $X_i=\theta +E_i$ for $n\ge 2$ independent and identically distributed observations, with $E_i$ having an exponential distribution of unknown scale $\beta >0$. Test the null hypothesis $\theta =0$ within $\theta \ge 0$. For nondegenerate data, the likelihood function isMaximizing over the scale gives $\hat{\beta }(\theta )=\overline{X}-\theta$, and the resulting profile likelihood increases with $\theta$. Thus the unrestricted maximum-likelihood estimates are $\hat{\theta }=X_{(1)}$ and $\hat{\beta }=\overline{X}-X_{(1)}$, while the null estimate is $\tilde{\beta }=\overline{X}$. The likelihood ratio is consequently

Under the null, put $A=n{X}_{(1)}$ and $D={\sum }_i(X_i-X_{(1)})$. The memoryless property gives independent gamma distributions of shapes $1$ and $n-1$ and common scale $\beta$. For a direct verification, fix which observation is the minimum and write its value as $z$ and the other observations as $z+e_j$. Their joint density is proportional to $e^{-nz/\beta }{\prod }_j{e}^{-e_j/\beta }$, which factors into an exponential density for the minimum and independent exponential densities for the excesses. The minimum's index does not affect the resulting laws of $A,D$.

By beta sampling by gamma ratios, $D/(A+D)\sim \mathrm{Beta}(n-1,1)$. ThereforeAn exact test size $\alpha$ is obtained by rejecting for $R\le {\alpha }^{n/(n-1)}$. Equivalently,Although the alternative adds one endpoint parameter, a regular one-parameter Wilks theorem calibration is incorrect: the support depends on the endpoint, and the null lies on its boundary. In particular, $-2\log R$ tends to a chi-squared distribution with two statistical degrees of freedom, not one.

Two equally numerous groups of independent counts with Poisson distributions have areas $a$ and $2a$. Under common density their group means are in ratio $1:2$. With group sums $S_1,S_2$ and $T=S_1+S_2$, the maximized alternative-to-null likelihood ratio isUse zero-count limits and $\Lambda =1$ when $T=0$. The regular Wilks theorem calibration has one degree of freedom. Exactly conditioning on $T$ gives $S_1\sim \mathrm{Binomial}(T,1/3)$ under the null; this supports small-count testing without the large-sample approximation.

The deviance of a fitted generalized linear model isFor regular nested models, the deviance reduction $D_0-D_1=2({\ell }_1-{\ell }_0)$ is asymptotically chi-squared with degrees of freedom equal to the difference in fitted dimensions.

The unscaled residual deviance of an exponential dispersion family compares its fitted mean with the saturated statistical model at a common dispersion parameter. Its normalization removes the factor $1/\varphi$ in the log-likelihood. In Poisson regression and binomial regression, $\varphi =1$, so it is twice the saturated-minus-fitted log-likelihood. Under suitable regularity, $D/\varphi$ has approximately a chi-squared distribution with the residual degrees of freedom; this is not an automatic accurate approximation for sparse responses. Nested deviance reductions with estimated dispersion lead to approximate F-tests after scaling by the dispersion estimate.

The null deviance is the unscaled deviance of the intercept-only mean model relative to the saturated model at a common dispersion parameter $\varphi$, retaining any known generalized linear model offset. It supplies a baseline against which predictor terms improve the fit. Its difference from a fitted model's residual deviance, divided by $\varphi$, is the statistic for a likelihood-ratio test of those added terms, with nuisance parameters handled consistently in both fits. In Poisson and binomial families, $\varphi =1$.

For independent unit-variance samples, testing equal means uses $Z=\sqrt{mn/(m+n)}(\bar{X}-\bar{Y})$ and rejects for large $|Z|$.

Likelihood-ratio tests of different null subspaces can have crossing rejection regions; comparing their quadratic-form geometry exhibits observations accepted by one and rejected by another.