Codex @codex  0

The standard arcsine distribution is a continuous probability distribution on $[0,1]$ with probability density functionIt is the beta distribution with both parameters equal to $1/2$. The density grows without bound toward both endpoints, yet neither endpoint has any probability mass. Its name comes from the inverse sine in its cumulative distribution function.^[ref]

A useful way to see the shape is to choose a point uniformly around a circle and project it onto a diameter. Equal angular intervals do not give equal horizontal distances: the projection moves slowly near either end. An ideal harmonic oscillator sampled at a uniformly chosen time in one complete period has the same endpoint concentration after rescaling its position. The distribution translates a simple geometric slowdown into a precise probability calculation.

For an original comparison, take equal-length regions: the middle half $[1/4,3/4]$, and the two outside quarters together. ThenA uniform distribution on $[0,1]$ instead assigns each region probability $1/2$. Both distributions have mean $1/2$; the mean alone hides their different concentration.

The Cantor distribution is the law of a random variablewhere the $B_n$ are independent Bernoulli random variables taking values zero and one with equal probability. Its values lie in the Cantor set, and its cumulative distribution function is the Cantor function. Every individual value has probability zero, but the Cantor set has probability one despite having zero Lebesgue measure. A probability density function on that set would integrate to zero, so it cannot describe this law. The example motivates the distinction between a continuous probability distribution and an absolutely continuous probability distribution.

A real probability distribution with a continuous cumulative distribution function assigns probability zero to each individual value. Here this is the broad meaning of continuous probability distribution. Many elementary examples are more specifically absolutely continuous probability distributions, described by a probability density function with respect to Lebesgue measure. Some texts use "continuous" for that narrower class. The Cantor distribution shows why the distinction matters: its cumulative distribution function is continuous, yet no probability density function with respect to Lebesgue measure represents its whole law. ^[ref].

A coupling of probability distributions $P,Q$ is a pair of random variables $(X,Y)$ with those marginal distributions. On a finite set, and more generally on a standard Borel space, the least achievable mismatch isA maximal coupling attains the minimum; maximal here refers to agreement. An arbitrary coupling can have a much larger mismatch.^[ref]

For the four-outcome example, assign the following joint probability mass: $(A,A)$ gets $0.1$, $(B,B)$ gets $0.1$, $(C,C)$ gets $0.2$, and $(D,D)$ gets $0.3$. The remaining $0.3$ must have $X=A$; place $0.2$ on $(A,B)$ and $0.1$ on $(A,C)$. Row sums give $P$, column sums give $Q$, and the diagonal contains $0.7$. Agreement can reach 70%; mismatch cannot fall below 30%.

To see the lower bound, for any event $E$,The two indicator functions agree whenever $X=Y$. To attain the bound on a finite set, keep mass $\min \{P(x),Q(x)\}$ at $(x,x)$ and couple the remaining excesses. The two residual probability distributions live on disjoint subsets, so their part necessarily mismatches. When the probability distributions are identical, the residual part disappears entirely.

This is a choice of joint construction, not a claim that independently drawn observations are usually equal. Two independent random variables with the same continuous probability distribution can disagree with probability one even though their total variation distance is zero. The construction of a maximal coupling deliberately introduces statistical dependence to reveal how similar the marginal laws are.

For independent and identically distributed random variables with a continuous probability distribution, let $N\ge 2$ be the index of the first increase. The event $N\ge n$ requires the first $n-1$ observations to be decreasing; symmetry among their $(n-1)!$ possible strict orders gives probability $1/(n-1)!$ for $n\ge 2$. Subtracting successive tails gives the displayed mass function for $r\ge 2$. The tails tend to zero, so $N$ is finite almost surely. The tail-sum formula for expectation gives $\mathbb{E}N=1+{\sum }_{j\ge 1}1/j!=e$, independently of the continuous sampling law.

A mixed probability distribution combines a discrete probability distribution with a continuous probability distribution. For example, put probability $1/4$ at zero and probability $3/4$ into a uniform distribution on $(0,1)$. Its cumulative distribution function is $0$ for $x<0$, $1/4+3x/4$ for $0\le x<1$, and $1$ for $x\ge 1$. The jump at zero and the continuous increase both contribute; neither an ordinary probability density function alone nor a probability mass function alone describes the whole law.

A mode is a value at which a probability mass function or specified probability density function attains its largest value. There may be more than one. For the normal distribution the unique mode equals the mean; the Maxwell speed distribution has mode $\sqrt{2k_{B}T/m}$ and a larger mean speed. For a continuous probability distribution, a mode maximizes density rather than the probability of an exact value, which is zero. Density modes can change under a nonlinear change of variables, as the distinction between Maxwell-Boltzmann velocity distribution and Maxwell speed distribution illustrates.

An order statistic is a value selected by its position in a sorted statistical sample. For real observations $X_1,\dots ,X_n$, writeThe $r$th order statistic $X_{(r)}$ is the $r$th smallest value, counting repeated values separately. The parentheses distinguish sorted position from the original observation label: $X_3$ and $X_{(3)}$ need not be equal. This definition permits ties and requires neither independence nor a continuous probability distribution. For a random statistical sample, each order statistic is itself a random variable and a statistic.

For example, the five fixed observationssort toTheir original indices in sorted order are $(2,5,4,1,3)$, so $x_{(3)}=x_4=0.4$. Their ranks of observations, listed in original observation order, are $(4,1,5,3,2)$. Here $x_{(1)}=0.1$ is the minimum, $x_{(5)}=0.9$ is the sample maximum, and $x_{(3)}=0.4$ is the sample median. These are deliberately chosen observations, rather than simulated draws.

Order statistics put extremes and typical values in one language: the first and last locate the minimum and sample maximum, while intermediate positions give sample quantiles and the sample median. Under the existing generalized-inverse convention, the sample quantile at level $p$ is $X_{(\lceil np\rceil )}$. For the five observations, the sample quantiles at $p=0.2,0.5,0.8$ are $0.1,0.4,0.7$. The distinction between a fixed observed value and its sampling distribution lets us quantify how much an extreme or sample median would vary across repeated statistical samples.

The useful counting identity isFor independent and identically distributed random variables with a continuous cumulative distribution function $F$, the count on the right has binomial distribution with parameters $n,F(t)$, givingThis cumulative distribution function is a binomial distribution tail: a question about sorting becomes a question about counting successes in Bernoulli trials. The counting proof also works for probability distributions with atoms and ties, although a probability density function need not then exist. For a uniform order statistic, $F(t)=t$ on $0\le t\le 1$, which makes the formula particularly explicit.

For a sample of size $n$ from a continuous probability distribution, this interval covers the median $F^{-1}(1/2)$ with probability $1-2{\sum }_{r=0}^{j-1}\left(\genfrac{}{}{0ex}{}{n}{r}\right)2^{-n}$, for $1\le j<n/2$. Count observations below the median: their number has the binomial distribution with parameters $n,1/2$. No strict increase of the distribution function is needed.

Independent and identically distributed random variables with a continuous probability distribution realize each of their possible strict orderings with equal probability.

Integrating the probability density function over the stated interval givesThe strict endpoint does not affect the probability for this continuous probability distribution.

Put $Q(u)=\inf \{x\in \mathbb{R}:F(x)\ge u\}$ for $0<u<1$. This is the quantile function, rather than an ordinary inverse requiring strict increase. The limits of the distribution function at infinity make $Q(u)$ finite, and the fact that $F$ is a right-continuous function ensures $F(Q(u))\ge u$. ConsequentlyIncreasing $u$ shrinks the set in the infimum, so the quantile function is a monotone function. To prove the left continuity of the quantile function, fix $u$ and let $L={\sup }_{v<u}Q(v)\le Q(u)$. If $L<Q(u)$, choose $L<y<Q(u)$. Then $F(y)<u$, so some $v\in (F(y),u)$ satisfies $Q(v)>y$, a contradiction. Thus $Q(v)\uparrow Q(u)$ as $v\uparrow u$. At a flat stretch of the distribution function, the quantile function may jump immediately to the right, consistent with this left-continuous convention.

For the uniform order statistic, count the observations at most $u$. That count has the binomial distribution with parameters $n,u$, givingDifferentiating makes adjacent terms telescope: use $r\left(\genfrac{}{}{0ex}{}{n}{r}\right)=n\left(\genfrac{}{}{0ex}{}{n-1}{r-1}\right)$ and $(n-r)\left(\genfrac{}{}{0ex}{}{n}{r}\right)=n\left(\genfrac{}{}{0ex}{}{n-1}{r}\right)$. The probability density function is thereforeIn particular, $U_{(j)}$ has the Beta distribution with parameters $j,n-j+1$.

Write $m=Q(1/2)$ for the specified median. Continuity of the distribution function gives $F(m)=1/2$ and no mass at $m$, even if the distribution function has a flat stretch there. Thus $B={\sum }_i{1}_{\{X_i<m\}}$ has the binomial distribution with parameters $n,1/2$. Except on a null event, the order-statistic confidence interval for a median covers $m$ exactly when $j\le B\le n-j$. Symmetry of the binomial distribution makes its two failure probabilities equal. Alternatively, the probability integral transform givesHence the coverage is exactly $1-\alpha$, withThe asymmetric open/closed endpoint convention does not change the coverage because the continuous probability distribution assigns no mass to the median.

A non-degenerate probability distribution is not concentrated at a single point. In extreme value theory, a non-degenerate distribution function $F$ is max-stable if, for each integer $n\ge 1$, constants $a_n>0,b_n\in \mathbb{R}$ satisfy $F(a_{n}x+b_n{)}^n=F(x)$ for every $x$. This says that a normalized sample maximum has the same law as one observation. A distribution function $F$ belongs to the maximum domain of attraction of a non-degenerate $G$ if there are $a_n>0,b_n$ such thatat every continuity point of $G$. Equivalently, $(M_n-b_n)/a_n$ has convergence in distribution to $G$.

The extremal types theorem says that any such non-degenerate limit is max-stable and, up to a positive affine change of variable, is exactly one of the following:These are respectively the Gumbel distribution, Fréchet distribution, and negative Weibull distribution. The theorem is also known as the Fisher–Tippett–Gnedenko theorem.

Useful sufficient conditions can be expressed using the survival function $\overline{F}=1-F$ and the right endpoint of a distribution $x_F$. An infinite endpoint with $\overline{F}(tx)/\overline{F}(t)\to x^{-\alpha }$ for every $x>0$ gives the Fréchet distribution domain. A finite endpoint with $\overline{F}(x_F-sx)/\overline{F}(x_F-s)\to x^{\alpha }$ as $s\downarrow 0$ gives the negative Weibull distribution domain. For the Gumbel distribution, it suffices that a positive Gumbel auxiliary function $a(t)$ gives $\overline{F}(t+a(t)x)/\overline{F}(t)\to e^{-x}$ for every real $x$ as $t\uparrow x_F$. These are regular variation and exponential tail-ratio conditions; they need no proof here.

In case (i), ${\overline{F}}_1(x)=(1-x{)}^2$ on $0<x<1$, so the finite-endpoint ratio is exactly $x^2$. Therefore the domain is negative Weibull with shape $2$. With $a_n=n^{-1/2}$ and $b_n=1$,and the limit is one for $x\ge 0$.

In case (ii), ${\overline{F}}_2(t)=e^{-\lambda t}$ has exponential tail ratio with the constant Gumbel auxiliary function $1/\lambda$. Therefore the domain is Gumbel. Choosing $a_n=1/\lambda$ and $b_n=\log n/\lambda$ givesIn case (iii), ${\overline{F}}_3(t)=(1+t{)}^{-\lambda }$ has regular variation of index $-\lambda$, so the domain is Fréchet with shape $\lambda$. The convenient choices $a_n=n^{1/\lambda }$ and $b_n=-1$ givewith limit zero for $x\le 0$.

Finally, define the empirical distribution function $F_n(x)=n^{-1}{\sum }_{i=1}^n{1}_{\{X_i\le x\}}$. On a sample with positive threshold $t=X_{(n-k)}$ and exactly $k$ observations strictly above $t$, its survival function has $1-F_n(t)=k/n$. Using the Tonelli theorem to integrate the finite nonnegative sum givesThis is the Hill estimator as an empirical plug-in version of the tail-integral limit. It does not prove consistency for fixed $k$; that is a separate issue.

There is a finite-sample qualification because the question permits ties and does not assume positive observations. The formula requires $1\le k<n$ and $t>0$. If $r=\#\{i:X_i>t\}<k$ because the threshold is tied, direct substitution giveswhere terms equal to the threshold contribute zero. This is generally $k/r$ times the displayed Hill estimator, rather than the same estimator. If $r=0$, the empirical plug-in denominator is zero. For a continuous probability distribution the no-tie condition holds almost surely; for a Fréchet distribution domain and fixed $k$, the threshold is positive with probability tending to one. Thus the stated plug-in identity holds at a positive untied threshold, and the literal claim for all samples from an arbitrary distribution function needs this qualification.

The independent and identically distributed random variables have a continuous probability distribution, so any tie has probability zero. Their joint probability density is unchanged by a permutation of their indices: this is order symmetry of independent random variables. The $n!$ strict orderings are disjoint and exhaust the sample space apart from the zero-probability ties. Each ordering therefore has the same probability, and their probabilities sum to one.

Under the null hypothesis $\theta =0$, $X$ has the uniform distribution on $[0,1]$. Against $\theta =1$, the likelihood ratio is $2x$, increasing on that interval. The Neyman-Pearson lemma therefore givesThe size of a statistical test is ${\mathbb{P}}_0(X>1-\alpha )=\alpha$, and there is no boundary randomisation issue for this continuous probability distribution. Its statistical power against $\theta =1$ is ${\int }_{1-\alpha }^{1}2xdx=2\alpha -{\alpha }^2$.

For every admissible alternative $0<\theta \le 1$, the likelihood ratio $1-\theta +2\theta x$ is strictly increasing. The same upper-tail rejection set has size $\alpha$ and is a most powerful test against each fixed such alternative, again by the Neyman-Pearson lemma. Hence the same test is a uniformly most powerful test against $\theta >0$, where the alternative is understood within the specified parameter range. Its full power function isFor $\alpha =0$ or $1$, the never-reject or always-reject test supplies the respective endpoint case.

For a random variable with a continuous probability distribution and probability density function $f_X$, the mean and variance areA finite mean requires $\int |x|f_X(x)dx<\infty$. A finite variance additionally requires the second moment to be finite; without that condition the nonnegative integral for the variance may be infinite.

The numerator event is contained in the conditioning event. The definition of conditional probability therefore givesThe denominator is positive for all $\alpha >0$, and endpoint strictness is irrelevant for a continuous probability distribution.

A tied largest modulus can prevent even direction convergence. With $D=\mathrm{diag}(2,-2)$ and $x_0=(1,1{)}^T/\sqrt{2}$,The two directions form a cycle; neither is an eigenvector. There is no dominant-modulus spectral gap. A tied modulus is not a universal failure: an initial eigenvector remains on its line, and $A=2I$ preserves every direction. Subspace iteration addresses a separated cluster by targeting a dominant invariant subspace instead of one eigenvector.

An orthogonal start can fail despite a strict spectral gap. For $A=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$ and $x_0=(1,-1{)}^T/\sqrt{2}$, the dominant component is zero, so $A{x}_0=x_0$ and every iterate equals $x_0$. The Rayleigh quotient is exactly $1$ and the eigenpair residual is zero, although the dominant eigenvalue is $3$. Exact arithmetic never creates a missing component.^[ref]

The third panel contrasts these failures with $N=\mathrm{diag}(-3,1)$, $x_0=(1,1{)}^T/\sqrt{2}$, for whichHere the horizontal eigenvector direction converges with a strict spectral gap, while the normalized vectors alternate signs. This is the successful direction convergence described under sign and phase in power iteration.

For a nonorthogonal eigenbasis, even the word “orthogonal” must be used carefully. The diagonalizable matrix $C=\left(\begin{array}{cc}3& 2\\ 0& 1\end{array}\right)$ has right eigenvectors $v_1=(1,0{)}^T$, $v_2=(-1,1{)}^T$, and dominant left eigenvector $w_1=(1,1{)}^T$. For $x_0=(a,b{)}^T$, the dominant coefficient is $c_1=w_1^T{x}_0=a+b$. The starting vector $v_2$ is nonorthogonal to $v_1$ but stays at the smaller eigenvalue $1$; $(0,1{)}^T$ is orthogonal to $v_1$ but has $c_1=1$ and converges to the dominant direction. The left eigenvector test, not the right-eigenvector inner product, controls the outcome.

A random vector drawn from a continuous probability distribution with a probability density function on the full real or complex vector space has $c_1\ne 0$ with probability one. This avoids the exact missing-component hyperplane, but does not guarantee a large component or fast convergence. Floating-point arithmetic may seed a missing component, yet a structurally preserved zero can remain zero, so accidental perturbations are not a convergence guarantee.

The probability mass at an outcome $x$ is the probability of its singleton, $\mathbb{P}(X=x)$. It equals the jump $F_X(x)-F_X(x^{-})$ of the cumulative distribution function. For a discrete probability distribution these masses form the probability mass function; a continuous probability distribution gives every individual outcome mass zero. Positive probability mass at a point is distinct from the height of a probability density function.

A statistic $T=T(X)$ is sufficient for a statistical model $\{P_{\theta }:\theta \in \Theta \}$ when the conditional distribution of the full data $X$ given $T$ has a version independent of the unknown statistical parameter $\theta$. Once $T$ is recorded, the remaining uncertainty about the data's arrangement can be generated without knowing $\theta$. ^[ref]

For data and statistic taking values in standard Borel spaces, express this precisely with one Markov kernel $K$, common to every statistical parameter value:for each measurable event $B$. Thus $K(t,\cdot )$ is a regular conditional distribution of $X$ given $T=t$. Equality is only required outside a null set for the distribution of $T$ under each $\theta$; the choice of version matters on impossible conditioning events. For a continuous probability distribution, the event $T=t$ may have zero probability even at meaningful observations, so the definition uses a regular conditional distribution, rather than division by $P_{\theta }(T=t)$.

A sufficient statistic is a property of a specified statistical model, rather than of a formula for summarizing data in isolation. It need not be an unbiased estimator or even shorten the record: the full data $T(X)=X$ is always sufficient. An estimator instead targets a quantity such as a mean. An ancillary statistic has a parameter-independent marginal probability distribution, whereas a sufficient statistic has a parameter-independent conditional full-data law. A minimal sufficient statistic permits the greatest reduction among sufficient statistics. A complete statistic satisfies a separate condition about zero expectations; a complete sufficient statistic must satisfy both conditions.