Gaussian covariance diagonality likelihood-ratio test Created 2026-10-14 Updated 2026-10-25
For independent observations from a nonsingular multivariate normal distribution with unknown mean, let . The unrestricted maximum-likelihood estimator of covariance is , while the diagonal restriction gives . The fitted trace terms both equal , so twice the log-likelihood difference is , where is the sample correlation matrix. Under the diagonal null hypothesis and fixed dimension, the Wilks theorem gives .
A logistic regression fitted to ungrouped Bernoulli observations has residual deviance . Comparing it automatically with is generally invalid: the saturated dimension grows with , with one observation per fitted probability. If the true model has only an intercept and success probability , then , whereas . Fixed-dimensional nested-model likelihood-ratio tests can still use Wilks theorem under regularity. For absolute fit, use meaningful grouping, residual diagnostics, or a fitted-model parametric bootstrap.
Let for independent and identically distributed observations, with having an exponential distribution of unknown scale . Test the null hypothesis within . For nondegenerate data, the likelihood function is
Maximizing over the scale gives , and the resulting profile likelihood increases with . Thus the unrestricted maximum-likelihood estimates are and , while the null estimate is . The likelihood ratio is consequently
Under the null, put and . The memoryless property gives independent gamma distributions of shapes and and common scale . For a direct verification, fix which observation is the minimum and write its value as and the other observations as . Their joint density is proportional to , which factors into an exponential density for the minimum and independent exponential densities for the excesses. The minimum's index does not affect the resulting laws of .
By beta sampling by gamma ratios, . Therefore
An exact test size is obtained by rejecting for . Equivalently,
Although the alternative adds one endpoint parameter, a regular one-parameter Wilks theorem calibration is incorrect: the support depends on the endpoint, and the null lies on its boundary. In particular, tends to a chi-squared distribution with two statistical degrees of freedom, not one.
For an independent normal distribution sample with unknown mean, test against unrestricted positive variance. Put and assume . Profiling the mean and variance gives the likelihood ratio for a nondegenerate sample. Its logarithmic derivative changes sign at , so small ratios select a lower and an upper tail.
Under the null hypothesis, Cochran's theorem gives . The exact size- cutoffs therefore satisfy
The second equation makes the two boundary ratios equal. Rejection on or is consequently the exact likelihood-ratio test, and its tails need not have equal probability. For a large sample, the Wilks theorem instead calibrates by . A Bartlett correction for a normal variance with unknown mean improves this asymptotic calibration; it is unnecessary when using the exact distribution.
For two independent statistical samples from exponential distributions with positive means and , write , and . The unconstrained maximum-likelihood estimators are and ; under equality of the means the common fitted mean is . Dividing the constrained maximum likelihood function by its unrestricted maximum gives the displayed likelihood ratio. Its maximum is one at , so the likelihood-ratio test rejects for sufficiently small values, in either tail of .
Under the null hypothesis, the sums have independent gamma distributions with a common rate, so by beta sampling by gamma ratios. This permits exact calibration. For large samples with , the Wilks theorem gives . Its upper-tail probability is the asymptotic p-value. The one statistical degree of freedom is the difference between the unrestricted and constrained parameter dimensions.
Likelihood-ratio test of area-proportional Poisson means Created 2026-10-06 Updated 2026-10-31
Two equally numerous groups of independent counts with Poisson distributions have areas and . Under common density their group means are in ratio . With group sums and , the maximized alternative-to-null likelihood ratio is
Use zero-count limits and when . The regular Wilks theorem calibration has one degree of freedom. Exactly conditioning on gives under the null; this supports small-count testing without the large-sample approximation.
For nested null and alternative parameter spaces, the likelihood-ratio test statistic is
Under the regularity conditions of Wilks theorem, its null limit is a chi-squared distribution whose degrees of freedom equal the difference in model dimensions.
Test the null hypothesis against the alternative hypothesis that the two rate parameters differ. Under , the pooled maximum-likelihood estimate is . Under the alternative, the estimates are and .
In the likelihood-ratio test statistic, the data-only terms cancel, and both fitted models have total . Consequently,
The alternative hypothesis adds one parameter, so Wilks theorem gives an approximate chi-squared distribution with one degree of freedom. Do not reject the common-rate model at the 5% level, since . The corresponding approximate p-value is . The fitted expected values are and days, but four observations per group supply little evidence that the underlying expected values differ; failure to reject does not establish equality. The chi-squared distribution calibration is asymptotic and should be treated cautiously for these small samples.
Under the diagonal covariance restriction, the profile log-likelihood separates into terms,
Differentiating each term gives the restricted maximum-likelihood estimator . Both fitted trace terms equal , so the likelihood-ratio test statistic is
where is the sample correlation matrix. The statistic is nonnegative by the Hadamard inequality; large values indicate dependence. Equivalently, the likelihood ratio is and rejection is for small values of this ratio.
For fixed and a positive-definite true covariance, the Wilks theorem says that twice the maximized log-likelihood difference for nested regular models converges under the null to a chi-squared distribution with degrees of freedom equal to the dimension difference. The unrestricted covariance has parameters and the restricted covariance has ; the unknown mean has the same parameters in both. Therefore the Gaussian covariance diagonality likelihood-ratio test of asymptotic level is
Within the multivariate normal distribution, diagonal covariance also means independent coordinates.
Multivariate analysis of variance tests differences between group mean vectors when each experimental unit has several correlated responses. For independent -dimensional observations in groups, use a common positive-definite covariance matrix and the multivariate normal model . The null hypothesis is equality of all means.
Write , group means , and the size-weighted grand mean . The within-group and between-group scatter matrices are
Expanding deviations about the group means yields total scatter , with zero cross terms. Under the null, orthogonal normal projections give independent Wishart distributions and .
With nonsingular , the Wilks lambda statistic is
where the nonnegative are eigenvalues of . Maximizing the common covariance under the two mean models gives the likelihood ratio , so small indicates large group separation relative to within-group spread. For fixed and large group sizes, the Wilks theorem gives under the null; a finite-sample reference distribution can also be used.
The lower-right sketch shows two elongated groups whose centroid difference lies nearly across, rather than along, their long axes. Their correlation makes this multivariate separation much clearer than either marginal mean shift alone. Nonsingular linear rescaling changes and by congruence, leaving the determinant ratio unchanged. MANOVA measures mean separation against the full within-group covariance geometry. Its interpretation requires independent observations and an adequate common-covariance model; separate coordinatewise tests discard that geometry and raise multiple-testing issues.
Partition the statistical parameter as , with of interest. For each fixed , maximize over the nuisance parameter to obtain . The profile likelihood and profile log-likelihood are
Their maximizer is the interest component of the joint maximum-likelihood estimator. At a regular nuisance maximum, and its negative Hessian matrix is the Schur complement
The profile likelihood ratio is unchanged by a one-to-one nuisance reparametrization, and transforms naturally under a one-to-one interest reparametrization. Under the regular conditions of Wilks theorem, has a limiting chi-squared distribution with dimension of degrees of freedom. Profiling is maximization, rather than integration or conditioning; need not be a normalized data density, and nuisance fitting can cause appreciable higher-order distortion.
The modified profile likelihood corrects for nuisance curvature and sample-coordinate distortion. In suitable coordinates , with an ancillary statistic, set
holding fixed when differentiating the data coordinates. Up to an interest-independent factor, the Barndorff-Nielsen modified profile likelihood is
Equivalently, implicit differentiation of the nuisance score equation gives , and
The semicolon denotes sample-coordinate differentiation, not a second parameter derivative. In orthogonal coordinates, the Cox-Reid adjusted profile likelihood uses the information adjustment alone; it agrees with this modification when , as happens below.
For the Inverse Gaussian distribution, write , and . Terms independent of both parameters can be absorbed into , giving
The nuisance score function is
It is positive below and negative above it. Thus its global maximum is
independent of . Put . Then
The nuisance observed information at this fit is
Since the constrained nuisance estimator is literally the full nuisance estimator as a function of the data, in the specified sample coordinates. Hence the modified profile likelihood for inverse Gaussian shape is
For and a nonconstant positive sample, the Cauchy-Schwarz inequality gives . The corresponding fits are and . A constant sample has and no finite profile maximum; under continuous independent sampling this event has probability zero when . In particular, the modified coefficient accounts for the fitted mean nuisance parameter.
For logistic regression, set and . Its Bernoulli log-likelihood, score function and Fisher information are
where . Solve by Fisher scoring or iteratively reweighted least squares. One scoring step is
Equivalently regress the working response on using weights . Iterate until the likelihood and coefficients stabilize, with rank and convergence checks. Separation in logistic regression can prevent a finite maximum: if a predictor direction strictly separates the zeros and ones, moving indefinitely along it improves the likelihood rather than producing a finite root.
Under the unrestricted model, each individual probability is estimated independently as . Every observed binary outcome then has probability one, so, using boundary limits,
The resulting residual deviance is . It is a legitimate likelihood contrast and can be used to compare fitted models. However, individual Bernoulli deviance need not have a chi-squared calibration: the saturated model has one boundary-valued parameter for each single observation, and its dimension grows with . The fixed-dimensional regularity behind Wilks theorem does not justify a reference here.
For a concrete demonstration, let the true model contain only an intercept and have success probability . Its fitted probability is , and
whereas a variable divided by converges to one. Thus treating the raw Bernoulli residual deviance as an ordinary absolute goodness-of-fit statistic can reject a correct model systematically. In contrast, deviance differences between fixed-dimensional nested logistic models have their usual asymptotic chi-squared reference under the relevant regularity assumptions. For absolute fit use meaningful replicated covariate groups and grouped-binomial logistic regression, or assess residual patterns and prediction on held-out data; a model-specific parametric bootstrap can calibrate a chosen statistic.
For an interest parameter and nuisance parameter , the profile likelihood is , where maximizes the likelihood at fixed . Its maximum occurs at the joint maximum-likelihood estimate of . It preserves the optimized likelihood ratios for different interest values and is unchanged by one-to-one reparametrization of the nuisance at each fixed . It generally is neither a marginal sampling density nor a conditional likelihood, and eliminating nuisance parameters by maximization can introduce appreciable small-sample distortion. Differentiating the constrained score equation gives the curvature formula
at a joint stationary point. The corresponding expected-information expression is the efficient information for the interest parameter. Under the regular conditions of Wilks theorem, twice the profile log-likelihood drop has an asymptotic chi-squared distribution with the dimension of the interest parameter as its degrees of freedom.
For scalar nuisance, the Barndorff-Nielsen modified profile likelihood may be written
where the sample-coordinate derivative holds and the ancillary statistic fixed. For vector nuisance the last two quantities are determinants. An equivalent form is , since differentiating the constrained nuisance score gives . The semicolon denotes differentiation in sample coordinates, not another ordinary parameter derivative. This Jacobian determinant adjustment is essential to the definition.
In the present model set
The log-likelihood is . Consequently
The profile score is , and its derivative is . The expression in parentheses is the variance of the fixed design values under positive weights proportional to . Thus the profile log-likelihood is strictly concave for a nonconstant design. Its score changes from to as goes from to , giving a unique finite fit. If all coincide, only the mean is identifiable and inference on alone is not defined; that exceptional design must be excluded.
The specified ancillary coordinates give . At fixed , therefore,
Meanwhile the constrained observed information is
Substitution proves
Equivalently the sample derivative of the nuisance score is , which equals at the constrained fit and gives the same cancellation. Thus the modified profile likelihood for exponential regression is proportional to the ordinary profile likelihood. This conclusion uses the given ancillary; no proof of its ancillarity is required.
The prose names a second year that differs from the table label. The calculations use the four supplied counts, interpreting the first and second rows as the two displayed periods; no numerical conclusion depends on resolving that label discrepancy.
The first generalized linear model treats the four counts as independent Poisson random variables, with a log link and both row and column regression factors. The row-by-column interaction makes the log-linear model saturated. With , write
Its four maximum-likelihood fitted values equal the observed counts. Thus , , , and
The interaction is the log odds ratio measuring association of row and column. Under independent Poisson sampling, the delta-method variances are for the intercept, for the row contrast, for the column contrast, and for the interaction. Their square roots reproduce the printed standard errors. Although the software labels the standardized ratios as t values, these are asymptotic normal Wald statistics, with known Poisson dispersion one, not exact finite-df Student tests.
The statistical saturated model has zero residual deviance and zero residual degrees of freedom; this is a perfect interpolation, not evidence of predictive adequacy. The null deviance compares the four observed counts with an intercept-only model, whose four fitted means are . Fisher scoring is the numerical score/information iteration used to fit the model; the iteration count is a convergence report, not a test statistic.
Dropping the interaction gives the independence log-linear model for a two-way contingency table, . Its Poisson regression margin-matching score equations match the row and column totals, so
Here and denote the observed row and column totals. In particular the fitted row multiplier is , the column multiplier is , and the intercept is . These produce the second coefficient table. Their standard errors follow from the inverse Fisher information matrix; for the row and column contrasts their squared values are and , while the intercept variance is .
The Poisson deviance relative to the saturated fit is
The linear terms cancel because fitted and observed totals agree. The additive model has three coefficients, leaving one degree of freedom. Wilks theorem calibrates this likelihood-ratio test approximately by , giving . The interaction Wald test instead gives ; the two are only asymptotically equivalent.
Fisher's exact test conditions on both margins under independence. If is the upper-left count, then
Its usual two-sided p-value sums the probabilities of all tables no more probable than the observed . That sum is , reproducing the output without an asymptotic approximation. It tests the same absence of row-column association, but conditioning and discreteness explain the numerical difference from the likelihood-ratio test.
In ordinary language, the male proportion among the cases is in the first row and in the second. The estimated case-sex odds ratio is , but its approximate 95-percent confidence interval is wide, . These small samples do not give convincing evidence that the sex composition of cases changed between the two periods. Failure to reject is not evidence that the compositions are exactly equal. Nor do these counts establish a male-to-female disease-incidence ratio or an incidence trend: population denominators and comparable ascertainment would be needed for those interpretations.
The first fit is a Bernoulli logistic-regression model for the probability of a positive response. Its fitted linear predictor is
and its fitted probability is . There are eight coefficients. The null degrees of freedom identify , and the residual degrees of freedom are .
Each slope is a change in the log odds per unit of its covariate, holding the others fixed. Exponentiating gives conditional odds ratios; for example a ten-unit glucose increase multiplies fitted odds by , and a ten-year age increase by . These are changes in odds, not multiplicative changes in probability, and the observational associations are not automatically causal effects. The intercept corresponds to all numerical covariates being zero, generally an extrapolation beyond meaningful subjects.
The standard errors come from the inverse Fisher information matrix of the logistic regression. With binomial dispersion parameter fixed at one, the displayed t ratios are asymptotic normal Wald statistics. In the full fit, glucose and the pedigree measure have clear individual evidence of nonzero slopes, with two-sided and . The body-mass and age terms are borderline, with and ; pregnancy count is weaker, . Blood pressure and skinfold have little individual evidence after adjustment. These labels should not be converted into a claim that the weaker predictors have no effect, or used as automatic model-selection rules.
A deviance residual here is
Its square is the observation's contribution to binomial deviance, so the squared residuals sum to the printed residual deviance. The five-number summaries describe the distribution of these signed residuals. The null deviance belongs to an intercept-only logistic regression; the full fitted deviance is . Four Fisher scoring iterations record convergence of the numerical likelihood fit.
The second model uses only glucose:
Its two coefficients leave residual degrees of freedom. A ten-unit glucose increase multiplies its fitted odds by . This differs from the adjusted multiplier , because the models condition on different covariates; both adjustment and noncollapsibility of the odds ratio can change logistic coefficients. The glucose-only model has deviance , so its maximized log-likelihood is lower than the full model's.
For these fixed-dimensional nested models, analysis of deviance and Wilks theorem give
Thus the additional six predictors collectively improve the glucose-only model, even though several individual Wald tests are not significant. These comparisons assume independent subjects, an appropriate logistic mean structure, full rank, and regular finite parameter estimates without separation.
It would be incorrect to infer that absolute fit is good merely because . Individual Bernoulli deviance need not have a chi-squared calibration: there is just one binary observation per saturated probability, and the saturated dimension increases with sample size. A simple counterexample is an intercept-only model with true probability , for which , not one. For absolute adequacy, inspect residual patterns and predicted-risk calibration, use meaningful grouped comparisons when appropriate, or calibrate a chosen fit statistic by parametric bootstrap. The regular nested-model tests above are not subject to that saturated-model dimensionality problem. Only the displayed summaries and a few example rows are supplied, so patient-level diagnostics or complete numerical refitting cannot be reconstructed from this page alone.
Under the unrestricted model, insert the maximum-likelihood estimates . Under the diagonal-covariance restriction, the quadratic form is minimized over the mean at , and the remaining objective separates over the diagonal entries:
Differentiating in gives ; the derivative changes from negative to positive there. Put . The fitted trace term is under either model, since . Consequently the likelihood-ratio test statistic is determined by
This is the Gaussian diagonal-covariance likelihood-ratio test. Reject for sufficiently small , equivalently for large . Under the regular large-sample calibration, Wilks theorem gives , since diagonality removes that many free covariance parameters. The exact critical constant can instead be obtained from the null distribution at the actual sample size.
For , write the sample correlation coefficient as . Then , so the likelihood-ratio test rejects for large , not merely for large positive . Under the null, the two variables have a bivariate normal distribution and are independent, and the exact test is
Indeed, condition on the centered first-variable observation vector. Decompose the second centered Gaussian vector into its component along that direction and the -dimensional orthogonal complement. Their squared lengths give independent and variables. Their ratio shows that has Student's t-distribution with statistical degrees of freedom.
Wilks theorem states that, under an interior null parameter and the usual identifiability, smoothness and nonsingular-information regularity conditions, for nested models converges under the null to a chi-square distribution with degrees of freedom equal to the difference of parameter dimensions. Boundary or nonidentifiable nulls need not have this distribution.
Write , and . The unrestricted maximum-likelihood estimates are and . Under the null the variance estimate is . Substituting both maxima into the Gaussian likelihood gives
This is for , with positive true variance; almost surely. The lost parameter dimension is one, even though variance is estimated in both models.
The two-sided Student t-test uses , where , and rejects when . Under the null has exactly the Student t-distribution with degrees of freedom. The likelihood-ratio statistic is
under the null, so its chi-square approximation agrees asymptotically with the squared asymptotic standard-normal t statistic. In fact the expression is strictly increasing in : with exact finite-sample calibration the tests have identical rejection regions for every , not just approximately for large samples.
For independent binomial distributions, the log-likelihood, up to known binomial coefficients, is
The saturated statistical model maximizes each term at . Under the logit link, the fitted probabilities are . Subtracting the fitted log-likelihood from the saturated log-likelihood gives the binomial deviance
A zero-count summand uses the continuous convention . For the restricted model, replace by to obtain . Since the saturated likelihood cancels, is the likelihood-ratio test statistic. For the nested restriction setting coefficients to zero, Wilks theorem gives the asymptotic chi-squared distribution under regularity and an interior true parameter; reject for a large difference.
For the six anther cells, gives a common success probability, gives a separate probability for each storage condition, gives a common intercept and a log-force slope, and gives storage-specific intercepts with a common log-force slope. Their parameter counts are , so their residual statistical degrees of freedom are . In , storage has a constant effect on log odds; multiplying force by multiplies the odds by in either storage group.
The relevant nested likelihood-ratio tests at the level are:
The two two-parameter models and are not nested, so their deviance difference has no ordinary nested chi-squared distribution calibration. The preferred parsimonious model is , with a storage effect and no log-force effect. Its deviance goodness-of-fit test is acceptable: on four statistical degrees of freedom. The common-probability model also passes its separate goodness-of-fit comparison, , but this does not contradict the relative evidence for adding storage: the two tests assess different null hypotheses. These are large-sample approximations; sufficiently large expected successes and failures, not merely the number of anthers, underpin their calibration.
For the log-transformed location-scale model, let denote the distribution function associated with . The change-of-variables formula for a probability density gives the untruncated density and distribution function
Consequently its upper-truncated likelihood is on the observed support, and the profile likelihood maximizes this over . Restricting to the corresponding subfamilies gives the gamma distribution and lognormal distribution profiles. Write the plotted likelihood ratio as
It measures how well each endpoint fits relative to that model's own best endpoint, after nuisance refitting. It is not a probability density function for and need not integrate to one. Endpoints whose ratios are near one are relatively compatible with the observations; a rapidly falling profile indicates more information favoring endpoints near the maximum.
All three profiles favor the smallest admissible endpoint, approximately 14 days; their information about larger endpoints differs substantially. Reading the actual PDF legend and curves, the dotted gamma distribution profile falls initially but is nearly flat around from about 18 days onward. The solid generalized log-gamma profile declines much further, to roughly at 19 days and at 21 days. The dashed lognormal distribution profile falls most strongly, to roughly at 18 days and at 21 days. These are graphical readings, not recomputed fits.
Thus the gamma distribution fit gives weak discrimination between moderate and very large endpoints: much of its untruncated mass can already be below these endpoints, making further changes to the truncation have little effect. Over the displayed range, the lognormal distribution fit gives stronger relative evidence against large endpoints, and the more flexible generalized log-gamma fit is intermediate. The solid curve remains close to the dashed curve near the maximum but has appreciably more relative support farther out. The plot shows slowing declines rather than a sharply determined finite upper endpoint; it cannot establish a biological maximum.
Normalization is essential to this comparison. Although the generalized family includes the two special families and therefore has at least as large a raw maximized likelihood at any fixed , its normalization constant can also be larger. Its normalized profile need not lie above the normalized profiles of its submodels. Their individual maxima have all been replaced by one, so this plot alone cannot compare overall goodness of fit or select a family while accounting for extra fitted parameters.
A set is a relative-likelihood support set once is chosen. Turning it into a calibrated confidence interval requires an appropriate sampling argument, simulation or other justified method. In particular, a usual Wilks theorem chi-squared cutoff cannot simply be assumed for this changing-support endpoint problem. The original incubation observations are not supplied here, so numerical refitting or calibrated limits cannot be recovered from the graph alone.
Writing , the likelihood function is and its supremum occurs at , including the limiting boundary cases zero and one. The likelihood ratio compares under the null with that supremum. Thus
Use at boundary outcomes. Wilks theorem gives asymptotic under the regular interior null, because one parameter is constrained. Directly, setting and expanding the bracket gives , hence
Under the null, and . The remainder vanishes in probability, proving the same chi-squared distribution limit. If an observed is not small, the quadratic numerical approximation is unreliable and the exact statistic should be used; exact binomial calibration is available. Persistent non-small deviations under a fixed alternative instead make the statistic grow linearly with , rather than having the null chi-squared distribution law.