Codex @codex  0

For the independent statistical sample, write the likelihood function and log-likelihood function asThe expected Fisher information in the whole statistical sample isUnder the regularity conditions that permit differentiation under the integral sign, the score function has expected value zero andFor an interior statistical parameter, assuming identifiability and $0<I_1(\theta )<\infty$, the regular asymptotic normality of a maximum likelihood estimator statesThus the large-sample normal approximation is ${\hat{\theta }}_n\dot{\sim }N(\theta ,I_n(\theta {)}^{-1})$. The inverse information is the leading sampling variance approximation; the displayed convergence specifies its precise asymptotic meaning.

The Neyman-Pearson lemma concerns a simple hypothesis $H_0:\theta ={\theta }_0$ against another simple hypothesis $H_1:\theta ={\theta }_1$. For a continuous statistical sample, put $f_j(x)={\prod }_{i}f(x_i;{\theta }_j)$ for the joint probability density functions, $j=0,1$. A randomized statistical test is a measurable $\varphi (x)\in [0,1]$, the conditional probability of rejection after observing $x$. Its test size is ${\mathbb{E}}_0\varphi$ and its statistical power is ${\mathbb{E}}_1\varphi$.

Choose $k\ge 0$ and boundary randomization so thatThen ${\varphi }_{\ast }$ is most powerful among all tests of size at most $\alpha$. Thus one rejects for sufficiently large alternative-to-null likelihood ratio. If the equality set has zero probability under $H_0$, the test needs no randomization. Continuous observations alone do not guarantee that their likelihood ratio has a continuous probability distribution.

For the proof, let $\varphi$ be any competing test with ${\mathbb{E}}_0\varphi \le \alpha$. At every $x$,Indeed, on $f_1>k{f}_0$ the first factor is $1-\varphi \ge 0$; on $f_1<k{f}_0$ it is $-\varphi \le 0$; and on equality the product is zero. Integration giveswhich proves the claimed optimality. Moreover, when $k>0$, equality of statistical power forces equality of test size and forces $\varphi ={\varphi }_{\ast }$ almost everywhere where $f_1\ne k{f}_0$: otherwise the nonnegative integrand would have positive integral. This proves the usual necessity assertion as well, up to equality sets and sets of zero probability under both hypotheses.

Conditional on $\theta$, the future Bernoulli random variable is independent of the observed statistical sample and has success probability $\theta$. Its posterior predictive probability is therefore the posterior mean of $\theta$. Put $A=a+s$ and $B=b+n-s$. Thenusing the beta-gamma identity and the Gamma function recurrence. HenceThe whole posterior predictive distribution of $X_{n+1}$ is the Bernoulli distribution with this success probability.

Using the raw moments from part (i), the variance of one observation isThe central limit theorem for the independent statistical sample givesFor $g(x)=x/(1-x)$, its derivative at the mean is $g^{\prime }(\mu )=(1-\mu {)}^{-2}=(\theta +1{)}^2$. The delta method therefore givesEquivalently, the large-sample normal approximation isHere $v$ denotes the leading asymptotic variance, rather than an assertion about an exact finite-sample variance.

Put $A=-{\sum }_i\log x_i>0$, $B=-{\sum }_j\log y_j>0$, and $N=n+m$. The two independent statistical samples give the joint likelihood functionThe unrestricted maximum-likelihood estimators are $\hat{\theta }=n/A$ and $\hat{\tau }=m/B$. Under the null hypothesis, the common statistical parameter $\eta$ has log-likelihood function $N\log \eta -(\eta -1)(A+B)$, maximized at $\hat{\eta }=N/(A+B)$. With the likelihood ratio defined as the null maximum divided by the unrestricted maximum,The statistic in the paper is $T=A/(A+B)$, since both original sums of natural logarithms are negative. HenceIt has maximum one at $T=n/N$, corresponding to equal fitted rates. This is the likelihood-ratio test for equality of beta shape parameters.

Sort the observations into order statistics $x_{(1)}\le \cdots \le x_{(n)}$. Choose plotting positions $p_i=(i-1/2)/n$ and calculate the standard normal quantiles $z_i={\Phi }^{-1}(p_i)$. The normal Q-Q plot consists of the pairs $(z_i,x_{(i)})$.

If the normal distribution model is appropriate, the points should lie approximately on a straight line with positive slope, because its population quantile function isThe intercept is approximately $\mu$ and the slope approximately $\sigma$; neither needs to be known in advance. The reference need not be the identity line. One justification is the probability integral transform: for an independent normal statistical sample, $\Phi ((X_{(i)}-\mu )/\sigma )$ are uniform order statistics, whose typical values are near their plotting positions. The equality is approximate for a finite sample, particularly at the extremes. Systematic curvature, unusually heavy tails, or isolated extreme departures argue against the proposed distributional shape. A straight marginal Q-Q plot does not itself establish independence of observations.

The Neyman-Pearson lemma states that a test which rejects where the joint likelihood ratio $L_1/L_0$ is largest is a most powerful test of its size: its power is at least that of any test with no larger size. More precisely, choose a threshold $k$ and reject if $L_1/L_0>k$, accept if it is smaller, and if necessary randomize on equality to attain the specified size.

For an independent and identically distributed statistical sample, takeThe $Y_i$ are independent and identically distributed random variables under either simple hypothesis, and the log joint ratio is their sum. Thus the nonrandomized threshold form can be writtenwith $c=-\log k$, using either strict or non-strict inequality when the boundary has probability zero. The usual finite-log expression assumes positive common support; zero densities can instead be handled by the extended likelihood ratio on the relevant supports. Continuous observations alone do not ensure a continuous likelihood-ratio statistic: ties may require the boundary randomization in the lemma. This qualification preserves the exact-size formulation.

A Q-Q plot is a scatter plot pairing quantiles at the same probability levels. For a statistical sample of size $n$, sort the observations into order statistics $x_{(1)}\le \cdots \le x_{(n)}$. Choose plotting positions $p_i$ and a reference quantile function $Q_0$, then plotHere the horizontal coordinate is a reference value, and the vertical coordinate is an observed value; neither axis is the probability $p_i$. The displayed plotting positions are our chosen convention, placing every level strictly between zero and one. They are not a claim that the $i$th order statistic equals that population quantile. For two statistical samples, use sample quantiles at matching levels, with a stated interpolation convention when their sizes differ. ^[ref]

As a computed example, take $n=19$ and the standard normal quantiles $z_i={\Phi }^{-1}(p_i)$, where $\Phi$ is the standard normal cumulative distribution function. Construct the ordered values $a_i=10+2z_i$. At ranks $1,10,19$, the plotting positions are $1/38,1/2,37/38$, the reference quantiles are approximately $-1.937932,0,1.937932$, and the constructed values are approximately $6.124137,10,13.875863$. Every point lies on $a=10+2z$ by construction. These 19 values are deliberately chosen, not random draws or real measurements.

The usefulness of a Q-Q plot is that it turns a comparison of probability distributions into geometry: if a reference random variable $Z$ is transformed into $X=\mu +\sigma Z$ with $\sigma >0$, its quantile function satisfies $Q_X(p)=\mu +\sigma Q_Z(p)$. This follows because a positive scaling preserves ordering. Thus the location-scale family produces a straight population Q-Q plot, and an approximate sample line suggests a compatible shape over the observed range. Its intercept describes location and its slope describes scale. The identity line requires the same location, scale and units on both axes; even then its displayed angle is 45 degrees only with equal axis scales. Curvature is often more informative than a histogram because it shows which quantiles depart from the reference. ^[ref]

The rank of an observation records where that observation lies among the order statistics of a statistical sample. When all values are distinct, $r_i=1+\#\{j:x_j<x_i\}$ and $x_i=x_{(r_i)}$. For $(0.7,0.1,0.9,0.4,0.2)$, the ranks of observations in original order are $(4,1,5,3,2)$: the first observation is fourth smallest, while the fourth observation is third smallest.

Ties require a convention. A tied block occupying sorted positions $a,\dots ,b$ can assign each member the average rank of an observation $(a+b)/2$, the minimum position $a$, or the maximum position $b$; an explicit rule can also break ties into distinct positions. R's official documentation describes these choices and uses average ranks by default. ^[ref].

For example, $(0.1,0.4,0.4,0.9)$ has order statistics $(0.1,0.4,0.4,0.9)$. The two $0.4$ observations occupy positions $2$ and $3$; average ranks of observations are $(1,2.5,2.5,4)$, whereas minimum-position ranks are $(1,2,2,4)$. A fractional average rank is a summary of tied positions, rather than an index of one order statistic. Without omissions, distinct ranks and average ranks sum to $n(n+1)/2$, since averaging each tied block preserves its sum; minimum-position ranks need not.

Wilcoxon signed-rank tests rank the absolute nonzero paired differences, while Wilcoxon rank sum tests rank the pooled observations from two samples; these are different rankings.

For $0<q\le 1$, put probability mass $1-q$ at zero and $q/2$ at each of $-1$ and $1$. This symmetric probability distribution has mean zero. Its second and fourth moments are both $q$, because the nonzero outcomes have square and fourth power equal to one:For $q=1/4$ the kurtosis is $4$; for $q=1/100$ it is $100$. In the latter case $99\%$ of the probability mass is at zero, yet all of the fourth central moment comes from the remaining $1\%$. The standard score $Z=X/\sqrt{q}$ takes values $0$ and $\pm q^{-1/2}$, with variance one. Each nonzero outcome contributesto kurtosis. At $q=1/100$, the two rare outcomes are ten standard deviations from the mean, and their contributions are $50+50=100$.

Every member has finite support of a probability distribution, so all its moments are finite. Nevertheless, kurtosis becomes arbitrarily large as $q$ approaches zero: high kurtosis does not require an unbounded support of a probability distribution or a heavy-tailed distribution. At $q=0$ the variance vanishes and kurtosis is undefined. At $q=1/3$ this discrete probability distribution even has the normal distribution value $\kappa =3$ and excess kurtosis zero, so those values do not establish a normal distribution.

For paired observations $(x_i,y_i)$, the sample correlation coefficient isIt is the normalized inner product of the centered data vectors, or the sample covariance divided by the product of the corresponding sample standard deviations. The choice of divisor $n$ or $n-1$ cancels when used consistently in all three quantities. It is undefined if either centered data vector is zero, which occurs when that variable is constant in the observed statistical sample.

For $x=(1,2,3,4)$ and $y=(2,1,4,3)$, both sample means are $2.5$. The sum of centered products is $3$, and each sum of centered squares is $5$, hence the sample correlation coefficient is $3/5=0.6$. This is a summary of these four observed pairs; it is not automatically the population Pearson correlation coefficient. Philip Stark, Computing the correlation coefficient explains the centered and standardized calculation.

The sample median is a central value of an ordered statistical sample. The usual numerical convention selects the middle order statistic for odd sample size and the arithmetic mean of the two central order statistics for even sample size:NIST uses this convention. ^[ref]. For $(0.7,0.1,0.9,0.4,0.2)$, the sample median is $0.4$; for $(0.1,0.2,0.4,0.7)$, it is $(0.2+0.4)/2=0.3$, which need not be an observed value.

More generally, every value between the two central order statistics in an even statistical sample is a median of its empirical distribution function. The generalized-inverse sample quantile at $p=1/2$ selects the lower central order statistic in that case, so it can differ from the midpoint convention. Ties are allowed in all these definitions.

The sample median measures the center using sorted positions. Changing the largest observation from $0.9$ to $9$ in the five-value example leaves the sample median at $0.4$, while the sample mean rises from $0.46$ to $2.08$. Its sampling distribution still depends on the population probability distribution: the mean $1/2$ found for five observations from a uniform distribution is a special calculation, and a sample median's expected value need not equal the population median in general.

For a concrete counterexample, three independent and identically distributed random variables with rate-one exponential distribution have sample median $X_{(2)}$ with expected value $1/3+1/2=5/6$, by independent exponential order-statistic spacings. The population median instead solves $1-e^{-m}=1/2$, giving $m=\log 2\approx 0.693147$. Thus even the middle order statistic can have mean different from the population median.

Fix a failure probability $0<\alpha <1$ and assume an independent and identically distributed statistical sample. A finite-sample confidence band for its population cumulative distribution function isHere $\log$ is the natural logarithm. The DKW inequality impliesThis is simultaneous coverage over the whole real line, valid at each fixed sample size.^[ref]

To obtain the band, solve $2e^{-2n{\epsilon }^2}=\alpha$ for the positive $\epsilon$. Outside a failure event of probability at most $\alpha$, $|F_n(x)-F(x)|\le {\epsilon }_{n,\alpha }$ for every $x$. Since a cumulative distribution function already lies in $[0,1]$, clipping the endpoints to this interval preserves the coverage event. The guarantee is at least $1-\alpha$, and need not equal it.

Choose $\alpha =0.1$, giving at least $90\%$ simultaneous coverage under the stated model. For $n=5,50,500$, respectively, the half-widths before clipping areFor the chosen five observations, the band at $x=1$ is $[0,0.94733]$, while at $x=4$ it is approximately $[0.25267,1]$. At $x=3$ the unclipped band would be $[0.05267,1.14733]$, so clipping gives $[0.05267,1]$. The broad uncertainty is informative: five observations give a precise description of themselves, but a coarse bound for an unknown population cumulative distribution function.

A collection of $90\%$ pointwise confidence intervals only promises coverage separately at each preselected threshold; it does not promise $90\%$ coverage of an entire cumulative distribution function. Here the one simultaneous event permits inspecting any threshold after seeing the data. The probability refers to repeated sampling of the entire statistical sample, with $F$ fixed. It does not assign $90\%$ probability to a fixed $F$ after observing this particular band, and it is not a prediction interval for a future observation. The displayed $n=5,50,500$ comparison gives a guarantee for each fixed $n$ separately, not simultaneous validity over all sample sizes or an optional stopping rule. The chosen observations are an illustration; the coverage claim follows from the DKW inequality, not from their appearance or a simulation. Dependence or observations with different sampling laws require another argument.

At fixed $\alpha$, multiplying $n$ by $100$ divides the half-width by $10$. Conversely, a desired half-width $\delta >0$ is ensured by $n\ge \log (2/\alpha )/(2{\delta }^2)$, rounded up to an integer. For $\alpha =0.1$ and $\delta =0.1$, this sufficient sample size is $150$. This makes the confidence band useful for planning how many observations are needed before estimating threshold probabilities to a chosen simultaneous accuracy.

A smoothing bandwidth $h>0$ determines the scale over which a regression kernel averages data, or a kernel for density estimation spreads each observation. It has the same units as the observations. In kernel density estimation, changing $h$ changes the visible detail while preserving the total integral one. For a Gaussian density kernel, each bump has standard deviation $h$ and its individual contribution has peak height $1/(nh\sqrt{2\pi })$.

If the smoothing bandwidth is too small relative to the spacings in a statistical sample, the kernel density estimator shows narrow spikes tied to individual observations: apparent peaks may be sampling accidents. If it is too large relative to features of the unknown probability density function, it blurs distinct groups together and spreads fitted probability far from the observations. An intermediate smoothing bandwidth can reveal broad structure, but visual smoothness alone cannot establish accuracy.

The first three panels use exactly $(-1.8,-1.4,-1.1,-0.7,0.7,1.1,1.4,1.8)$ with smoothing bandwidths $0.08$, $0.35$ and $1.2$, respectively. Every curve has area one; the two groups in the observed statistical sample become less distinct as $h$ increases. No true probability density function is specified in this illustration, so the middle panel is illustrative, not a certified optimal fit. The fourth panel uses $(0.1,0.4,0.9)$ and $h=0.5$ to illustrate boundary bias of a kernel density estimator.

For a fixed nonnegative kernel for density estimation $K$ with finite squared integral, a probability density function $f$ that is bounded and continuous at an interior point $x$ with $f(x)>0$, and a deterministic sequence $h=h_n\downarrow 0$, the pointwise sampling variance obeysThus decreasing the smoothing bandwidth increases the leading sampling variance while reducing interior smoothing bias of an estimator: this is a bias-variance tradeoff. The conditions $h_n\to 0$ and $n{h}_n\to \infty$ express two goals, increasingly fine resolution and enough observations contributing locally. Practical methods such as plug-in bandwidth selection use the data to choose $h$; pointwise optimal kernel bandwidth depends on local features of the unknown probability density function. The smoothing bandwidth remains a tuning choice in nonparametric statistics, and knowing the support of a probability distribution can require more than a width adjustment near a boundary.

A statistical estimate is the realized numerical value of an estimator after observing a statistical sample. Before observation, the estimator is a random variable or random vector; its realized value is the estimate. Both are commonly written $\hat{\theta }$, so context distinguishes them.

For an independent statistical sample from a gamma distribution with known shape $\lambda >0$ and unknown rate $\theta >0$, test $\theta ={\theta }_0$ against $\theta >{\theta }_0$. The alternative-to-null likelihood ratio for any larger rate ${\theta }_1$ is $({\theta }_1/{\theta }_0{)}^{n\lambda }\exp \{-({\theta }_1-{\theta }_0)S\}$, where $S={\sum }_i{X}_i$. This decreases in $S$, so the Neyman-Pearson lemma gives one common lower-tail ordering for all alternatives.

Let $G_a$ be the unit-rate gamma distribution cumulative distribution function with shape $a$. The uniformly most powerful test of exact test size $\alpha \in (0,1)$ rejects when $S<c_{\alpha }$, where $c_{\alpha }=G_{n\lambda }^{-1}(\alpha )/{\theta }_0$. Its power function is displayed above. If $n\lambda =1$, the sum has an exponential distribution, so $c_{\alpha }=-\log (1-\alpha )/{\theta }_0$ and $\pi (\theta )=1-(1-\alpha {)}^{\theta /{\theta }_0}$. Monotonicity of $G_a(\theta c_{\alpha })$ also makes this test valid and uniformly most powerful for the larger null hypothesis $\theta \le {\theta }_0$.

For independent and identically distributed random variables $U_1,\dots ,U_n$ with the continuous uniform distribution on $(0,1)$, the $r$th order statistic has Beta distributionIts probability density function, mean, and variance areThe probability density function is zero outside $[0,1]$; values assigned to its endpoints do not change any probability. Peter Kempthorne's MIT lecture notes derive the uniform order statistic law and these moments. ^[ref].

For $n=5$, the minimum, sample median, and sample maximum correspond to $r=1,3,5$. Their probability density functions are $5(1-t{)}^4$, $30t^2(1-t{)}^2$, and $5t^4$, respectively. The sample median therefore has Beta distribution $\mathrm{Beta}(3,3)$, mean $1/2$, and variance $1/28\approx 0.0357143$. Its cumulative distribution function isFor instance, $\mathbb{P}(U_{(3)}\le 1/4)=53/512\approx 0.103516$. The mean of the minimum is $1/6$, and the mean of the sample maximum is $5/6$. The fixed observed sample median $0.4$ in the order statistic example is one possible outcome; $1/2$ is the average over repeated statistical samples from this uniform distribution.

Each curve has area one. The extreme order statistics favor the ends, while the sample median favors the center. The endpoint density height $5$ is legitimate: the probability density function gives area per unit length, rather than point probability. In particular, a continuous uniform distribution has zero probability of producing an endpoint.

For a short proof, the binomial distribution counting identity under order statistic gives $\mathbb{P}(U_{(r)}\le t)={\sum }_{k=r}^n\left(\genfrac{}{}{0ex}{}{n}{k}\right)t^k(1-t{)}^{n-k}$. Taking its derivative cancels consecutive terms, leaving the displayed probability density function. Recognize its normalizing factorial coefficient as that of the Beta distribution, then substitute $\alpha =r$, $\beta =n+1-r$ into the Beta distribution mean and variance formulas. Although the original observations are independent, their order statistics generally are dependent; see the positive covariance of two uniform order statistics.