Likelihood-ratio test

A likelihood-ratio test rejects where the alternative likelihood is sufficiently large relative to the null likelihood.

For two independent statistical samples from exponential distributions with positive means $\theta$ and $\mu$, write $A={\sum }_{i=1}^n{X}_i$, $B={\sum }_{j=1}^m{Y}_j$ and $T=A/(A+B)$. The unconstrained maximum-likelihood estimators are $A/n$ and $B/m$; under equality of the means the common fitted mean is $(A+B)/(n+m)$. Dividing the constrained maximum likelihood function by its unrestricted maximum gives the displayed likelihood ratio. Its maximum is one at $T=n/(n+m)$, so the likelihood-ratio test rejects for sufficiently small values, in either tail of $T$.

Under the null hypothesis, the sums have independent gamma distributions with a common rate, so $T\sim \mathrm{Beta}(n,m)$ by beta sampling by gamma ratios. This permits exact calibration. For large samples with $n/(n+m)\to \rho \in (0,1)$, the Wilks theorem gives $-2\log \Lambda \stackrel{d}{\to }{\chi }_1^2$. Its upper-tail probability is the asymptotic p-value. The one statistical degree of freedom is the difference between the unrestricted and constrained parameter dimensions.

Take two independent statistical samples from beta distributions with parameters $(\theta ,1)$ and $(\tau ,1)$, of positive sizes $n,m$. Put $A=-{\sum }_i\log X_i$, $B=-{\sum }_j\log Y_j$, and $T=A/(A+B)$. To test the null hypothesis $\theta =\tau$, divide the maximized common-rate likelihood function by the unrestricted maximum to obtain the displayed likelihood ratio. Indeed the unrestricted fitted rates are $n/A,m/B$, and the common fitted rate is $(n+m)/(A+B)$; substitution cancels the exponential factors.

Under the null hypothesis, $A$ and $B$ have independent gamma distributions with a common rate. Beta-gamma independence gives $T\sim \mathrm{Beta}(n,m)$. The derivative of $\log (T^n(1-T{)}^m)$ changes sign at $n/(n+m)$, so the exact critical region of size $\alpha$ is $T<k_1$ or $T>k_2$, with $k_1<n/(n+m)<k_2$. Its cutoffs satisfyHere $F_{n,m}$ is the beta distribution cumulative distribution function. Equal likelihood ratios at the cutoffs do not generally mean equal tail probabilities. Increasing the ratio threshold continuously adds positive null distribution mass on both sides, proving uniqueness for $0<\alpha <1$.

For an independent normal distribution sample with unknown mean, test ${\sigma }^2={\sigma }_0^2$ against unrestricted positive variance. Put $q={\sum }_i(x_i-\bar{x}{)}^2/{\sigma }_0^2$ and assume $n\ge 2$. Profiling the mean and variance gives the likelihood ratio $\lambda (q)=(q/n{)}^{n/2}{e}^{(n-q)/2}$ for a nondegenerate sample. Its logarithmic derivative $\frac{1}{2}(n/q-1)$ changes sign at $n$, so small ratios select a lower and an upper tail.

Under the null hypothesis, Cochran's theorem gives $Q\sim {\chi }_{n-1}^2$. The exact size-$\alpha$ cutoffs $0<q_1<n<q_2$ therefore satisfyThe second equation makes the two boundary ratios equal. Rejection on $q\le q_1$ or $q\ge q_2$ is consequently the exact likelihood-ratio test, and its tails need not have equal probability. For a large sample, the Wilks theorem instead calibrates $W=-2\log \lambda$ by ${\chi }_1^2$. A Bartlett correction for a normal variance with unknown mean improves this asymptotic calibration; it is unnecessary when using the exact $Q$ distribution.

For independent observations from a nonsingular multivariate normal distribution with unknown mean, let $S=n^{-1}{\sum }_i(y_i-\bar{y})(y_i-\bar{y}{)}^T$. The unrestricted maximum-likelihood estimator of covariance is $S$, while the diagonal restriction gives $D=\mathrm{diag}(S_{11},\dots ,S_{pp})$. The fitted trace terms both equal $p$, so twice the log-likelihood difference is $W=n\log (\det D/\det S)=-n\log \det R$, where $R$ is the sample correlation matrix. Under the diagonal null hypothesis and fixed dimension, the Wilks theorem gives $W\Rightarrow {\chi }_{p(p-1)/2}^2$.

For independent multivariate normal observations with unrestricted mean and positive-definite sample covariance matrix $S=n^{-1}{\sum }_i(x_i-\bar{x})(x_i-\bar{x}{)}^T$, the unrestricted maximum-likelihood estimates are $\bar{x},S$. Requiring a diagonal population covariance matrix gives the same mean and covariance estimate $D=\mathrm{diag}(S_{11},\dots ,S_{pp})$. Both fitted quadratic terms equal $np$, so their likelihood quotient is $(|S|/|D|{)}^{n/2}=|R{|}^{n/2}$, where $R=D^{-1/2}S{D}^{-1/2}$ is the sample correlation matrix. Small determinants give evidence against diagonality. For two variables this is a two-sided test of their sample correlation.

For independent samples from normal distributions with separate unknown means, test equal variances against ${\sigma }_X^2>{\sigma }_Y^2$. Profiling the means gives sample means. Under the null the variance maximum is $(S_X+S_Y)/(n_X+n_Y)$; the separate maxima are $S_X/n_X$ and $S_Y/n_Y$. If the latter obey the proposed order, their likelihood quotient isIt decreases strictly as $(S_X/n_X)/(S_Y/n_Y)>1$ increases. Otherwise the ordered alternative maximum lies on the equal-variance boundary and the quotient is one. Under the null, Cochran's theorem gives independent chi-squared residual sums with degrees $n_X-1,n_Y-1$, so rejection is in the upper tail of the displayed F-distribution statistic, calibrated at the desired significance level.

For independent unit-variance Gaussian samples with means ${\mu }_X,{\mu }_Y$, testing ${\mu }_X\ge A,{\mu }_Y=0$ against unrestricted means gives $T=-2\log \Lambda =n[{\bar{Y}}^2+(A-\bar{X}{)}_{+}^2]$. The unrestricted maximum-likelihood estimators are the two sample means, while the null estimators are $\max (A,\bar{X})$ and zero. Under the null, write $W=\sqrt{n}(\bar{X}-A)=W_0+\delta$, $\delta =\sqrt{n}({\mu }_X-A)\ge 0$, and $Z=\sqrt{n}\bar{Y}$, with $W_0,Z$ independent standard Gaussian variables. Then $T=Z^2+(-W_0-\delta {)}_{+}^2$ decreases pointwise with $\delta$, so the largest rejection probability occurs at the boundary ${\mu }_X=A$. There its law is the chi-bar-squared distribution $\frac{1}{2}{\chi }_1^2+\frac{1}{2}{\chi }_2^2$. The exact size-$\alpha$ threshold $c_{\alpha }$ is therefore determined by $1-\Phi (\sqrt{c_{\alpha }})+\frac{1}{2}{e}^{-c_{\alpha }/2}=\alpha$.

For a scalar parameter restricted to $\psi \ge 0$, test $\psi =0$ against $\psi >0$ with identifiable interior nuisance parameters. A locally quadratic regular likelihood has an efficient null score $Z\sim N(0,1)$, and the constrained fit projects the unconstrained local maximizer onto the nonnegative half-line. ConsequentlyFor a positive statistic, its asymptotic upper-tail p-value is half the ordinary one-degree chi-squared tail. Its 5% critical value is the 90th chi-squared percentile, approximately 2.7055. Additional boundary nuisance parameters or nonidentified mixture parameters can change this law.

Testing a single nonnegative random-effect variance against zero puts the null on the boundary of the parameter space. In the usual regular increasing-independent-groups limit, the likelihood-ratio test statistic has limit $\frac{1}{2}{\delta }_0+\frac{1}{2}{\chi }_1^2$, rather than ${\chi }_1^2$. A finite-sample simulation calibrated for the actual design avoids relying on this asymptotic approximation.

With fixed $X,Z$, compare $N(X\beta ,{\sigma }^{2}I)$ and $N(X\beta ,{\sigma }^{2}I+{\tau }^{2}Z{Z}^T)$ by maximizing the ordinary likelihood function in both models. Their likelihood-ratio test statistic is unchanged by $Y\mapsto Xb+cY$, $c>0$, because both maximized log-likelihoods change by the same $-n\log c$. Its null law can therefore be simulated using independent $N(0,I)$ responses. Comparing the observed statistic with simulated statistics by $(1+\#\{T_b\ge T_{obs}\})/(B+1)$ gives a conservative finite-sample Monte Carlo p-value, subject to correctly maximizing both likelihoods.

For two parameter values ${\theta }_1$ and ${\theta }_0$, the likelihood ratio is $f(x\mid {\theta }_1)/f(x\mid {\theta }_0)$.

A log-likelihood ratio compares two likelihoods on an additive scale. For independent observations its increments add. Under distributions $P_0,P_1$ with suitable integrability and common support, $E_0[\log (d{P}_1/d{P}_0)]=-D(P_0\Vert P_1)\le 0$ and $E_1[\log (d{P}_1/d{P}_0)]=D(P_1\Vert P_0)\ge 0$, where $D$ is Kullback-Leibler divergence. This sign change motivates likelihood-based sequential monitoring.

For nested null and alternative parameter spaces, the likelihood-ratio test statistic isUnder the regularity conditions of Wilks theorem, its null limit is a chi-squared distribution whose degrees of freedom equal the difference in model dimensions.

A Bartlett correction rescales a regular likelihood-ratio test statistic so that its null expected value matches its limiting chi-squared degrees of freedom to a higher asymptotic order. If ${\mathbb{E}}_{0}W=d(1+b/n+O(n^{-2}))$, divide $W$ by $1+b/n$. The operation uses a model-specific Bartlett correction coefficient; matching the mean alone is not a general proof of any claimed distributional error rate.

If ${\mathbb{E}}_{0}W=d(1+b/n+O(n^{-2}))$, the Bartlett-corrected statistic has mean $d+O(n^{-2})$. It is commonly calibrated against a chi-squared distribution with $d$ degrees of freedom. Equivalently, to the displayed order, one may multiply $W$ by $1-b/n$.

The coefficient $b$ is the leading relative null-mean discrepancy of a regular likelihood-ratio test statistic. It is distinct from the finite-sample divisor $1+b/n$ used in a Bartlett correction. Different conventions call either of these a correction factor, so the normalization should always be stated explicitly.

For a sample of size $n\ge 2$ from a normal distribution with unknown mean, the null variance likelihood-ratio test statistic is $W=n[-\log (U/n)+U/n-1]$, where $U\sim {\chi }_{n-1}^2$. With $V=(U-n)/n$, its expected Taylor terms start with $n[\mathbb{E}{V}^2/2-\mathbb{E}{V}^3/3+\mathbb{E}{V}^4/4]$. Here $\mathbb{E}{V}^2=(2n-1)/n^2$, $\mathbb{E}{V}^3=(2n-3)/n^3$, and $\mathbb{E}{V}^4=(12n^2+4n-15)/n^4$. Higher fixed-order terms contribute only $O(n^{-2})$ under valid termwise asymptotic integration. Thus ${\mathbb{E}}_{0}W=1+11/(6n)+O(n^{-2})$ and $W_B=W/[1+11/(6n)]$.

A generalized likelihood-ratio test compares the likelihood maximized under the null with the likelihood maximized over the full parameter space.

Let $X_i=\theta +E_i$ for $n\ge 2$ independent and identically distributed observations, with $E_i$ having an exponential distribution of unknown scale $\beta >0$. Test the null hypothesis $\theta =0$ within $\theta \ge 0$. For nondegenerate data, the likelihood function isMaximizing over the scale gives $\hat{\beta }(\theta )=\overline{X}-\theta$, and the resulting profile likelihood increases with $\theta$. Thus the unrestricted maximum-likelihood estimates are $\hat{\theta }=X_{(1)}$ and $\hat{\beta }=\overline{X}-X_{(1)}$, while the null estimate is $\tilde{\beta }=\overline{X}$. The likelihood ratio is consequently

Under the null, put $A=n{X}_{(1)}$ and $D={\sum }_i(X_i-X_{(1)})$. The memoryless property gives independent gamma distributions of shapes $1$ and $n-1$ and common scale $\beta$. For a direct verification, fix which observation is the minimum and write its value as $z$ and the other observations as $z+e_j$. Their joint density is proportional to $e^{-nz/\beta }{\prod }_j{e}^{-e_j/\beta }$, which factors into an exponential density for the minimum and independent exponential densities for the excesses. The minimum's index does not affect the resulting laws of $A,D$.

By beta sampling by gamma ratios, $D/(A+D)\sim \mathrm{Beta}(n-1,1)$. ThereforeAn exact test size $\alpha$ is obtained by rejecting for $R\le {\alpha }^{n/(n-1)}$. Equivalently,Although the alternative adds one endpoint parameter, a regular one-parameter Wilks theorem calibration is incorrect: the support depends on the endpoint, and the null lies on its boundary. In particular, $-2\log R$ tends to a chi-squared distribution with two statistical degrees of freedom, not one.

Two equally numerous groups of independent counts with Poisson distributions have areas $a$ and $2a$. Under common density their group means are in ratio $1:2$. With group sums $S_1,S_2$ and $T=S_1+S_2$, the maximized alternative-to-null likelihood ratio isUse zero-count limits and $\Lambda =1$ when $T=0$. The regular Wilks theorem calibration has one degree of freedom. Exactly conditioning on $T$ gives $S_1\sim \mathrm{Binomial}(T,1/3)$ under the null; this supports small-count testing without the large-sample approximation.

The deviance of a fitted generalized linear model isFor regular nested models, the deviance reduction $D_0-D_1=2({\ell }_1-{\ell }_0)$ is asymptotically chi-squared with degrees of freedom equal to the difference in fitted dimensions.

The unscaled residual deviance of an exponential dispersion family compares its fitted mean with the saturated statistical model at a common dispersion parameter. Its normalization removes the factor $1/\varphi$ in the log-likelihood. In Poisson regression and binomial regression, $\varphi =1$, so it is twice the saturated-minus-fitted log-likelihood. Under suitable regularity, $D/\varphi$ has approximately a chi-squared distribution with the residual degrees of freedom; this is not an automatic accurate approximation for sparse responses. Nested deviance reductions with estimated dispersion lead to approximate F-tests after scaling by the dispersion estimate.

The null deviance is the unscaled deviance of the intercept-only mean model relative to the saturated model at a common dispersion parameter $\varphi$, retaining any known generalized linear model offset. It supplies a baseline against which predictor terms improve the fit. Its difference from a fitted model's residual deviance, divided by $\varphi$, is the statistic for a likelihood-ratio test of those added terms, with nuisance parameters handled consistently in both fits. In Poisson and binomial families, $\varphi =1$.

For independent unit-variance samples, testing equal means uses $Z=\sqrt{mn/(m+n)}(\bar{X}-\bar{Y})$ and rejects for large $|Z|$.

Likelihood-ratio tests of different null subspaces can have crossing rejection regions; comparing their quadratic-form geometry exhibits observations accepted by one and rejected by another.