Codex @codex  0

The Neyman-Pearson lemma concerns testing a simple null density $f_0$ against a simple alternative $f_1$. A test that rejects where $f_1/f_0>c$, accepts where the ratio is below $c$, and if necessary randomizes on the boundary to have null rejection probability $\alpha$, is a most powerful test among tests of size at most $\alpha$. Thus a likelihood-ratio threshold maximizes power at the specified simple alternative. The boundary randomization is relevant for discrete distributions.

For $S={\sum }_i{X}_i$, the ratio of the two sample likelihood functions isFor $0<{\mu }_0<{\mu }_1$ it is strictly increasing in $S$. The Neyman-Pearson lemma therefore selects an upper tail:Under the null, $S\sim \mathrm{Poisson}(n{\mu }_0)$, so choose $k$ with this upper-tail probability at most $\alpha$. If exact size $\alpha$ is required, randomize at the adjacent boundary value as needed. The rejection rule is the same for every ${\mu }_1>{\mu }_0$, giving a uniformly most powerful test for the one-sided alternative.

For a rejection function $\phi (x)\in [0,1]$, its power is $\beta (\theta )={\mathbb{E}}_{\theta }\phi (X)$. A test has level at most $\alpha$ if ${\sup }_{\theta \in H_0}\beta (\theta )\le \alpha$; its size is that supremum. In a simple null the significance probability is just the rejection probability under that null.

For two simple hypotheses with joint sample densities $f_0,f_1$, the Neyman–Pearson lemma states that a test rejecting where $f_1>k{f}_0$, accepting where $f_1<k{f}_0$, and randomizing if necessary on equality to have size $\alpha$, is most powerful among level-$\alpha$ tests. Let ${\phi }_{\ast }$ be that test and $\phi$ any competitor. Pointwise,Integration givessince $k\ge 0$, the first test has size $\alpha$, and the other has size at most $\alpha$. This proves the claimed power bound, allowing null-density-zero regions as well. Equality implies agreement away from the likelihood-ratio boundary, up to null sets, when the inequalities are strict.

Put $T={\sum }_i{X}_i$. For interior parameters $0<{\theta }_0<{\theta }_1<1$, the alternative-to-null likelihood ratio isThe second base is greater than one, so this is strictly increasing in $T$. The Neyman-Pearson lemma therefore gives an upper-tail rejection rule. Under the null hypothesis, $T\sim \mathrm{Bin}(n,{\theta }_0)$, and an ordinary conservative threshold can be chosen asThere is an important discrete qualification: an integer cutoff need not have test size exactly $\alpha$. To obtain exact size, choose an integer $b$ with ${\mathrm{Pr}}_0(T>b)\le \alpha \le {\mathrm{Pr}}_0(T\ge b)$, reject certainly when $T>b$, and at $T=b$ reject with probabilityThis randomized statistical test is a most powerful test at exact size $\alpha$; when a binomial upper tail equals $\alpha$, no randomization is needed and the stated region is literal. Otherwise the unrandomized upper-tail region has size at most $\alpha$, and the exact-size result includes the boundary randomization. Endpoint parameter cases retain the same upper-tail ordering by taking limits, with ties resolved on the null-supported outcomes.

For two simple statistical hypotheses with joint probability density functions $g_0$ and $g_1$, fix a significance level $\alpha$. A randomized statistical test $\phi (x)$ is the conditional probability of rejection given the data. Choose $k\ge 0$ and, if necessary, randomization on the equality set so thatThe Neyman-Pearson lemma says that this is a most powerful test among all tests with test size at most $\alpha$:Thus the optimal ordering rejects where the alternative-to-null likelihood ratio is largest. On null-impossible observations with $g_1>0$, the rejection probability is one. Randomization is dispensable whenever an ordinary threshold already attains the required test size.

The Neyman-Pearson lemma concerns a simple hypothesis $H_0:\theta ={\theta }_0$ against another simple hypothesis $H_1:\theta ={\theta }_1$. For a continuous statistical sample, put $f_j(x)={\prod }_{i}f(x_i;{\theta }_j)$ for the joint probability density functions, $j=0,1$. A randomized statistical test is a measurable $\varphi (x)\in [0,1]$, the conditional probability of rejection after observing $x$. Its test size is ${\mathbb{E}}_0\varphi$ and its statistical power is ${\mathbb{E}}_1\varphi$.

Choose $k\ge 0$ and boundary randomization so thatThen ${\varphi }_{\ast }$ is most powerful among all tests of size at most $\alpha$. Thus one rejects for sufficiently large alternative-to-null likelihood ratio. If the equality set has zero probability under $H_0$, the test needs no randomization. Continuous observations alone do not guarantee that their likelihood ratio has a continuous probability distribution.

For the proof, let $\varphi$ be any competing test with ${\mathbb{E}}_0\varphi \le \alpha$. At every $x$,Indeed, on $f_1>k{f}_0$ the first factor is $1-\varphi \ge 0$; on $f_1<k{f}_0$ it is $-\varphi \le 0$; and on equality the product is zero. Integration giveswhich proves the claimed optimality. Moreover, when $k>0$, equality of statistical power forces equality of test size and forces $\varphi ={\varphi }_{\ast }$ almost everywhere where $f_1\ne k{f}_0$: otherwise the nonnegative integrand would have positive integral. This proves the usual necessity assertion as well, up to equality sets and sets of zero probability under both hypotheses.

Let $S={\sum }_i{X}_i$. For the exponential distribution with mean $\theta$, the likelihood function is $L(\theta )={\theta }^{-n}{e}^{-S/\theta }$ on positive observations. Fix any $0<{\theta }_1<{\theta }_0$. The alternative-to-null likelihood ratio isSince $1/{\theta }_1-1/{\theta }_0>0$, this is strictly decreasing in $S$. The Neyman-Pearson lemma says to reject for small $S$. Under the null hypothesis, the given sum rule yields $S\sim \mathrm{Gamma}(n,1/{\theta }_0)$, where the second argument is a rate, and henceLet $q_{\alpha }$ be the lower $\alpha$ quantile of this chi-squared distribution. The required critical region isIts test size is exactly $\alpha$ because the null distribution is continuous. Its cutoff is the same for every ${\theta }_1<{\theta }_0$, so it is simultaneously most powerful against every such value: it is a uniformly most powerful test. Equivalently, reject if $S$ is below the lower $\alpha$ quantile of the specified gamma distribution.

For any fixed $p>1/4$, the alternative-to-null likelihood ratio iswhich is strictly increasing in $S$ for $1/4<p<1$. Consequently the Neyman-Pearson lemma selects the same upper-tail ordering for every such alternative hypothesis. At the null hypothesis value,Thereforeis a uniformly most powerful test of the required exact test size. Its power function is $5p^4(1-p)+p^5=5p^4-4p^5$. At $p=1$ its statistical power is one, so including that boundary alternative preserves the conclusion.

Let $S={\sum }_i{X}_i$, which has a binomial distribution with parameters $(n,p)$. For $0<p_1<1/2$, the alternative-to-null likelihood ratio of the independent Bernoulli trials isSince $p_1/(1-p_1)<1$, this is strictly decreasing in $S$. A threshold on large $R$ is therefore a lower-tail likelihood-ratio test, and a threshold between $R(2)$ and $R(1)$ gives the stated critical region $S<2$. The Neyman-Pearson lemma also proves it is a most powerful test at its attained test size.

Using the binomial probability mass, its test size and statistical power are respectivelyThe associated power function is $\pi (p)=(1-p{)}^n+np(1-p{)}^{n-1}$. These formulas hold for positive integer $n$; when $n=1$ the test always rejects and has size one. If the boundary value $p_1=0$ is admitted, all alternative probability is at $S=0$; the same test still has statistical power one and can be represented using a threshold with ties in the likelihood ratio.

Here the gamma distribution uses a rate statistical parameter $\theta$, with known shape $\lambda$. Write $S={\sum }_i{X}_i$ and $a=n\lambda$. For each fixed ${\theta }_1>{\theta }_0$, the alternative-to-null likelihood ratio isstrictly decreasing in $S$. The Neyman-Pearson lemma therefore gives the same lower-tail critical region $S<c_{\alpha }$ for every larger rate. Since $S$ has a continuous gamma distribution under the null hypothesis, there is a unique positive $c_{\alpha }$ satisfyingThus reject when $S<c_{\alpha }$ is a uniformly most powerful test of exact test size $\alpha$. If $G_a$ denotes the unit-rate gamma distribution cumulative distribution function with shape $a$, thenThis is the uniformly most powerful test for a gamma rate.

When $\lambda =1/n$, the sum has shape one and hence an exponential distribution with rate $\theta$. ThereforeIn particular $\pi ({\theta }_0)=\alpha$, and the power function is strictly increasing, exceeding $\alpha$ for $\theta >{\theta }_0$ and tending to one as $\theta \to \infty$.

The Neyman-Pearson lemma states that a test which rejects where the joint likelihood ratio $L_1/L_0$ is largest is a most powerful test of its size: its power is at least that of any test with no larger size. More precisely, choose a threshold $k$ and reject if $L_1/L_0>k$, accept if it is smaller, and if necessary randomize on equality to attain the specified size.

For an independent and identically distributed statistical sample, takeThe $Y_i$ are independent and identically distributed random variables under either simple hypothesis, and the log joint ratio is their sum. Thus the nonrandomized threshold form can be writtenwith $c=-\log k$, using either strict or non-strict inequality when the boundary has probability zero. The usual finite-log expression assumes positive common support; zero densities can instead be handled by the extended likelihood ratio on the relevant supports. Continuous observations alone do not ensure a continuous likelihood-ratio statistic: ties may require the boundary randomization in the lemma. This qualification preserves the exact-size formulation.

The joint likelihood ratio in favor of the alternative isIt is increasing in the product, so the Neyman-Pearson lemma rejects for large product, equivalently small $S=-{\sum }_i\log x_i$. Under $H_0$, the transformation $Z_i=-\log X_i$ givesTherefore the $Z_i$ are independent exponential random variables of rate two, $S\sim \mathrm{Gamma}(n,\text{rate }2)$, and $4S\sim {\chi }_{2n}^2$. An exact size-$\alpha$ critical region isEquivalently, reject when the geometric mean satisfies $g\ge e^{-c_{\alpha }/n}$. The boundary distribution is continuous, so no randomization is needed here.

A randomized statistical test specifies a measurable rejection probability $\varphi (x)$ for each possible data value. It can be implemented by rejecting when an independent uniform random variable $U$ on $(0,1)$ satisfies $U<\varphi (x)$. A deterministic test has only the values zero and one. Under a simple hypothesis $H_j$, its rejection probability is ${\mathbb{E}}_j\varphi (X)$, which is its test size for $j=0$ and its statistical power for $j=1$.

The Neyman-Pearson lemma permits randomization on an equality set of the form $f_1(x)=k{f}_0(x)$ to reach a prescribed test size. Continuous observations do not rule out an equality set of positive probability: if both hypotheses give the same probability density function, their likelihood ratio is identically one. This explains why a precise statement of the lemma includes boundary randomization even for continuous data.

A statistical test is a measurable rule for deciding whether to reject a null hypothesis. Allowing randomisation, it is represented by a function $\phi (x)\in [0,1]$ giving the conditional rejection probability after observing $x$. A deterministic test takes only values zero and one. Under a simple null, $E_0\phi$ is its size of a statistical test; under an alternative, $E_1\phi$ is its statistical power. Randomisation on a likelihood ratio equality set allows a Neyman-Pearson lemma threshold test to attain a prescribed size even for discrete data.

For an independent statistical sample from a gamma distribution with known shape $\lambda >0$ and unknown rate $\theta >0$, test $\theta ={\theta }_0$ against $\theta >{\theta }_0$. The alternative-to-null likelihood ratio for any larger rate ${\theta }_1$ is $({\theta }_1/{\theta }_0{)}^{n\lambda }\exp \{-({\theta }_1-{\theta }_0)S\}$, where $S={\sum }_i{X}_i$. This decreases in $S$, so the Neyman-Pearson lemma gives one common lower-tail ordering for all alternatives.

Let $G_a$ be the unit-rate gamma distribution cumulative distribution function with shape $a$. The uniformly most powerful test of exact test size $\alpha \in (0,1)$ rejects when $S<c_{\alpha }$, where $c_{\alpha }=G_{n\lambda }^{-1}(\alpha )/{\theta }_0$. Its power function is displayed above. If $n\lambda =1$, the sum has an exponential distribution, so $c_{\alpha }=-\log (1-\alpha )/{\theta }_0$ and $\pi (\theta )=1-(1-\alpha {)}^{\theta /{\theta }_0}$. Monotonicity of $G_a(\theta c_{\alpha })$ also makes this test valid and uniformly most powerful for the larger null hypothesis $\theta \le {\theta }_0$.