Put . For interior parameters , the alternative-to-null likelihood ratio isThe second base is greater than one, so this is strictly increasing in . The Neyman-Pearson lemma therefore gives an upper-tail rejection rule. Under the null hypothesis, , and an ordinary conservative threshold can be chosen asThere is an important discrete qualification: an integer cutoff need not have test size exactly . To obtain exact size, choose an integer with , reject certainly when , and at reject with probabilityThis randomized statistical test is a most powerful test at exact size ; when a binomial upper tail equals , no randomization is needed and the stated region is literal. Otherwise the unrandomized upper-tail region has size at most , and the exact-size result includes the boundary randomization. Endpoint parameter cases retain the same upper-tail ordering by taking limits, with ties resolved on the null-supported outcomes.
For a random sample, write , , for its joint probability density functions. Let be any competing statistical test of level at most . Pointwise,Indeed, where , ; where , , so both factors have the same sign. On the equality set the product is zero, regardless of randomization. Integrate over the joint sample space using its ordinary volume measure:Since and ,This proves maximal power. A continuous sample distribution does not itself guarantee that the equality set of the likelihood ratio has zero probability; allowing randomization there makes the proof valid in that case too.
For two simple statistical hypotheses with joint probability density functions and , fix a significance level . A randomized statistical test is the conditional probability of rejection given the data. Choose and, if necessary, randomization on the equality set so thatThe Neyman-Pearson lemma says that this is a most powerful test among all tests with test size at most :Thus the optimal ordering rejects where the alternative-to-null likelihood ratio is largest. On null-impossible observations with , the rejection probability is one. Randomization is dispensable whenever an ordinary threshold already attains the required test size.
Let be the likelihood function, with full parameter space and null hypothesis . The generalized likelihood-ratio test compares the best null fit to the best unrestricted fit:The likelihood ratio lies between zero and one, and small values count against the null hypothesis; equivalently, large values of the likelihood-ratio test statistic count against it.
Compute restricted and unrestricted maximum-likelihood estimates, evaluate their log-likelihoods, and form . Under the regular conditions of Wilks theorem, the null limit isThese statistical degrees of freedom count independent parameter restrictions. The regularity includes a locally identifiable smooth model, nonsingular Fisher information, and an interior null value; parameter-dependent support can invalidate the result.
The count vector has a multinomial distribution, with likelihood functionMaximize its log-likelihood using a Lagrange multiplier for the sum constraint. The equations and give and . Since all counts are positive and the log-likelihood is strictly concave on the interior of the probability simplex, this is the global maximum-likelihood estimate.
Under the uniform null hypothesis, each category has expected countThe common multinomial coefficient cancels from the likelihood ratio:Taking minus twice its logarithm gives exactlyThere are free category probabilities, because their sum is one, and the null fixes all of them. Thus its regular asymptotic chi-squared distribution has statistical degrees of freedom.
Write , and . For nondegenerate data from the shifted exponential distribution, the likelihood function isFor fixed feasible , differentiating its log-likelihood with respect to givesThe derivative changes from positive to negative there. Substituting gives the profile likelihood , which increases with . Therefore the unrestricted maximum-likelihood estimates arewhile under the null hypothesis the estimate is . Their likelihood ratio isSince the likelihood-ratio test rejects for small , the requested form is obtained withThe formulas assume , which holds almost surely for a continuous sample with ; coincident observations give an unbounded unrestricted likelihood and can be treated by its supremum.
An exact calibration is available and also explains why Wilks theorem should not be applied here. Under the null, the memoryless property makes and independent gamma distributions with common scale and shapes and . By beta sampling by gamma ratios,Consequently, for ,This yields an exact test size , independent of the unknown scale. The support endpoint and boundary null make this a likelihood-ratio test for an exponential support endpoint, with a different calibration from a regular one-parameter test.
The Neyman-Pearson lemma concerns a simple hypothesis against another simple hypothesis . For a continuous statistical sample, put for the joint probability density functions, . A randomized statistical test is a measurable , the conditional probability of rejection after observing . Its test size is and its statistical power is .
Choose and boundary randomization so thatThen is most powerful among all tests of size at most . Thus one rejects for sufficiently large alternative-to-null likelihood ratio. If the equality set has zero probability under , the test needs no randomization. Continuous observations alone do not guarantee that their likelihood ratio has a continuous probability distribution.
For the proof, let be any competing test with . At every ,Indeed, on the first factor is ; on it is ; and on equality the product is zero. Integration giveswhich proves the claimed optimality. Moreover, when , equality of statistical power forces equality of test size and forces almost everywhere where : otherwise the nonnegative integrand would have positive integral. This proves the usual necessity assertion as well, up to equality sets and sets of zero probability under both hypotheses.
Let . For the exponential distribution with mean , the likelihood function is on positive observations. Fix any . The alternative-to-null likelihood ratio isSince , this is strictly decreasing in . The Neyman-Pearson lemma says to reject for small . Under the null hypothesis, the given sum rule yields , where the second argument is a rate, and henceLet be the lower quantile of this chi-squared distribution. The required critical region isIts test size is exactly because the null distribution is continuous. Its cutoff is the same for every , so it is simultaneously most powerful against every such value: it is a uniformly most powerful test. Equivalently, reject if is below the lower quantile of the specified gamma distribution.
Put , and . Independence gives the likelihood functionThe unrestricted maximum-likelihood estimators are and . Under the null hypothesis, the common mean maximizes at . These are maxima because each one-parameter score function has the sign of its observed sum minus its sample size times its proposed mean.
Define the generalized likelihood ratio to be the constrained maximum divided by the unrestricted maximum. The exponential factors cancel, givingSince and ,It has maximum one at , when the two fitted means agree, and is small toward either endpoint. A likelihood-ratio test rejects for small , or equivalently large . This is the likelihood-ratio test for equality of exponential means.
A likelihood-ratio test rejects for small . The derivative of the natural logarithm of its variable factor isThus increases up to and decreases thereafter, tending to zero at both endpoints. Every nontrivial lower level set of therefore consists of two tails. For a prescribed test size , the exact critical region isWriting for the cumulative distribution function of the beta distribution with parameters , the required two equations areThe first makes both cutoffs have the same likelihood ratio; the second calibrates the null hypothesis rejection probability. Equivalently, the second left-hand side isAs the common likelihood ratio threshold increases from zero to one, this probability increases continuously and strictly from zero to one; hence the equations determine unique cutoffs. The two tail probabilities need not each equal when . Exact calibration holds for every common rate because the given null distribution of is independent of that rate. Boundary equalities have zero probability.
Put , , and . The two independent statistical samples give the joint likelihood functionThe unrestricted maximum-likelihood estimators are and . Under the null hypothesis, the common statistical parameter has log-likelihood function , maximized at . With the likelihood ratio defined as the null maximum divided by the unrestricted maximum,The statistic in the paper is , since both original sums of natural logarithms are negative. HenceIt has maximum one at , corresponding to equal fitted rates. This is the likelihood-ratio test for equality of beta shape parameters.
For any fixed , the alternative-to-null likelihood ratio iswhich is strictly increasing in for . Consequently the Neyman-Pearson lemma selects the same upper-tail ordering for every such alternative hypothesis. At the null hypothesis value,Thereforeis a uniformly most powerful test of the required exact test size. Its power function is . At its statistical power is one, so including that boundary alternative preserves the conclusion.
Let , which has a binomial distribution with parameters . For , the alternative-to-null likelihood ratio of the independent Bernoulli trials isSince , this is strictly decreasing in . A threshold on large is therefore a lower-tail likelihood-ratio test, and a threshold between and gives the stated critical region . The Neyman-Pearson lemma also proves it is a most powerful test at its attained test size.
Using the binomial probability mass, its test size and statistical power are respectivelyThe associated power function is . These formulas hold for positive integer ; when the test always rejects and has size one. If the boundary value is admitted, all alternative probability is at ; the same test still has statistical power one and can be represented using a threshold with ties in the likelihood ratio.
Here the gamma distribution uses a rate statistical parameter , with known shape . Write and . For each fixed , the alternative-to-null likelihood ratio isstrictly decreasing in . The Neyman-Pearson lemma therefore gives the same lower-tail critical region for every larger rate. Since has a continuous gamma distribution under the null hypothesis, there is a unique positive satisfyingThus reject when is a uniformly most powerful test of exact test size . If denotes the unit-rate gamma distribution cumulative distribution function with shape , thenThis is the uniformly most powerful test for a gamma rate.
When , the sum has shape one and hence an exponential distribution with rate . ThereforeIn particular , and the power function is strictly increasing, exceeding for and tending to one as .
The derivative of the log ratio is , so increases from zero to one on and decreases to zero on . Rejecting for a sufficiently small ratio therefore gives two tails with .
Under , Cochran's theorem gives the exact sampling distribution , independently of the unknown mean. Writing for its cumulative distribution function, the two cutoffs are determined byThe first equation calibrates the exact size of a statistical test. The second is , so both tails belong to one likelihood ratio cutoff. Hence the exact critical region is . Its two tail probabilities are not generally equal: equal-tailed chi-squared cutoffs give a different test.
The Neyman-Pearson lemma states that a test which rejects where the joint likelihood ratio is largest is a most powerful test of its size: its power is at least that of any test with no larger size. More precisely, choose a threshold and reject if , accept if it is smaller, and if necessary randomize on equality to attain the specified size.
For an independent and identically distributed statistical sample, takeThe are independent and identically distributed random variables under either simple hypothesis, and the log joint ratio is their sum. Thus the nonrandomized threshold form can be writtenwith , using either strict or non-strict inequality when the boundary has probability zero. The usual finite-log expression assumes positive common support; zero densities can instead be handled by the extended likelihood ratio on the relevant supports. Continuous observations alone do not ensure a continuous likelihood-ratio statistic: ties may require the boundary randomization in the lemma. This qualification preserves the exact-size formulation.
The joint likelihood ratio in favor of the alternative isIt is increasing in the product, so the Neyman-Pearson lemma rejects for large product, equivalently small . Under , the transformation givesTherefore the are independent exponential random variables of rate two, , and . An exact size- critical region isEquivalently, reject when the geometric mean satisfies . The boundary distribution is continuous, so no randomization is needed here.
Profile log-likelihood 2026-10-14
The logarithm of the profile likelihood is the full log-likelihood evaluated at the nuisance fit constrained to the specified parameter of interest. It preserves likelihood ratios and maximizers, but is not generally a normalized sampling log-density. Conditioning and integrating out nuisance parameters are different operations.
Randomized statistical test 2026-10-25
A randomized statistical test specifies a measurable rejection probability for each possible data value. It can be implemented by rejecting when an independent uniform random variable on satisfies . A deterministic test has only the values zero and one. Under a simple hypothesis , its rejection probability is , which is its test size for and its statistical power for .
The Neyman-Pearson lemma permits randomization on an equality set of the form to reach a prescribed test size. Continuous observations do not rule out an equality set of positive probability: if both hypotheses give the same probability density function, their likelihood ratio is identically one. This explains why a precise statement of the lemma includes boundary randomization even for continuous data.
Second moment of a mixture likelihood ratio Created 2026-10-06 Updated 2026-10-07
If and the nonnegative mixture weights sum to one, the likelihood ratio of the mixture model is . Expanding proves the displayed chi-squared divergence identity. For independent random variables sampled from a fixed component, one-observation cross moments multiply across observations. The component is drawn once for the whole sample; redrawing it per observation gives a different mixture model.