Codex @codex  0

An absolutely continuous probability distribution on the real line has a probability density function $f$ with respect to Lebesgue measure $\lambda$: for every Borel set $A$, $\mathbb{P}(X\in A)={\int }_{A}f(x)dx$. Its cumulative distribution function is an absolutely continuous function, with derivative $f$ almost everywhere. The probability density function and cumulative distribution function example under probability distribution illustrates this class.

For a finite, time-homogeneous Discrete-time Markov chain with row-stochastic transition matrix $P$, a state $a$ is absorbing exactly when $P_{aa}=1$. Nonnegativity and the row sum then force $P_{aj}=0$ for $j\ne a$: once entered, $a$ is never left. Equivalently, $\{a\}$ is a closed communicating class. A larger closed communicating class may keep moving internally; closure means confinement to a set, whereas absorption at a state means remaining at one point.

An absorbing state is useful for recording a terminal outcome: after a toy routine finishes, keep its outcome in the state space so that the same transition matrix still describes subsequent steps. Calling a finite Markov chain absorbing usually also requires that some absorbing state be accessible from every state. Merely containing an absorbing state is insufficient: another closed communicating class could prevent absorption. See Grinstead and Snell, Introduction to Probability, section 11.2 for this convention.

Consider a constructed routine with unfinished modes $u,v$ and terminal outcomes $A,B$. All the following transition probabilities are chosen for illustration, not estimated from observations. In state order $(u,v,A,B)$, takeThe communicating class $\{u,v\}$ is open, while $\{A\}$ and $\{B\}$ are closed communicating classes consisting of absorbing states. Both terminal outcomes are accessible from both unfinished modes, sometimes by an indirect path.

Let $T=\inf \{n\ge 0:X_n\in \{A,B\}\}$ be the terminal hitting time. Write $h_i={\mathbb{P}}_i(X_T=A)$ for the absorption probability and $t_i={\mathbb{E}}_{i}T$ for the expected hitting time, with the subscript denoting the starting state. First-step analysis givesHere $h_A=1$, $h_B=0$, and $t_A=t_B=0$. Solving these two pairs of linear equations yieldsStarting at $u$ therefore ends at $A$ with probability $0.6$ and takes $3.2$ steps on average to reach either terminal state. The terminal hitting time counts the transition into the outcome, but no later self-transitions. Outcome and duration answer different questions: from $v$, outcome $A$ is less likely, yet termination is faster on average.

The matrix power $P^2$ gives the probability distribution after two steps from $u$ asThus ${\mathbb{P}}_u(T>2)=1/2$, and the probability of having already reached $A$ is $3/8$, still below its eventual absorption probability $3/5$. From either unfinished mode, each next step has terminal probability at least $1/4$. Consequently ${\mathbb{P}}_i(T>n)\le (3/4{)}^n$, proving termination almost surely and finite expected hitting time directly. The Fundamental matrix of a finite absorbing Markov chain packages both calculations and counts visits to unfinished modes.

If independent random variables $X_n$ converge almost surely to a finite $X$, then ${\mathrm{limsup}}_n{X}_n$ is measurable for the tail sigma-algebra. The Kolmogorov zero-one law forces every rational threshold event for that limit to have probability zero or one. A real-valued probability distribution with this property is a point mass, so $X$ is almost surely constant. Consequently the Second Borel-Cantelli lemma upgrades almost sure convergence of independent sequences to complete convergence.

At parameter $1/2$, primal and dual bond percolation on the square lattice have the same probability distribution. Suppose an infinite percolation cluster exists. Large boxes meet it with probability tending to one. The square-root trick for positively associated events and quarter-turn symmetry imply that each side has an infinite exterior open arm with probability tending to one. The same holds for dual arms. A union bound then gives positive probability of primal top and bottom arms together with dual left and right arms.

Use a primal box $[-r,r{]}^2$ and a dual box with sides at $\pm (r+1/2)$. Open only primal edges whose endpoints both lie in the primal box, preserving all outward and exterior arm edges. The two primal arms are joined inside. Planarity prevents the two alternating dual arms from joining: a proposed dual connection together with the inner boundary arc separates one primal infinite arm from infinity. Thus the modified configuration has two infinite dual clusters. Its positive probability by finite modification of Bernoulli percolation contradicts uniqueness of the infinite percolation cluster. Consequently the percolation probability at $1/2$ is zero. This is often called Zhang's argument.

Amortized analysis bounds the total cost of every permitted sequence of operations from a specified initial state. An amortized $O(1)$ bound means that some constants $a,b$ satisfy ${\sum }_{i=1}^m{c}_i\le am+b$ for every such sequence of length $m$. The cost model and allowed operations are part of the claim; Big O notation alone does not specify them.

This distinction, and the credit and potential method viewpoints, are developed in R. Endre Tarjan's 1985 paper Amortized Computational Complexity, sections 1–2, pp. 306–308. The dynamic array doubling example below shows how an expensive operation can pay for itself through the cheap operations that precede it.

An amortized analysis can bound the total time complexity $T(m,n)$ of every allowed sequence of $m$ operations by $ma(n)+I(n)$, where $n$ is a size parameter and $I(n)$ accounts for initialization or the specified initial state. The bound $a(n)$ is then an amortized per-operation time complexity. Dividing the total bound by $m$ explains the averaging; it is a guarantee over every permitted sequence, without an assumed probability distribution on requests. Individual operations can cost more than $a(n)$.

A statistic is ancillary for a statistical model if its probability distribution is the same for every parameter value. Its marginal law carries no information about the parameter, but its observed value can identify the precision or configuration of the experiment. Fisherian conditional inference uses the conditional distribution of an informative statistic given the observed ancillary value. Ancillarity does not mean that the ancillary is independent of every informative statistic.

Choose $\Theta$ with uniform distribution on $(0,\pi /2)$ and set $X={\sin }^2\Theta$. Since the sine is increasing on this interval,Differentiation gives $f(x)=1/(\pi \sqrt{x(1-x)})$, and $F(0)=0$, $F(1)=1$ prove normalization. This also evaluates the density's improper integral without treating its divergent endpoint values as infinite probability.

For the circle construction, instead choose $\Phi$ with uniform distribution on $[0,2\pi )$ and put $Y=\cos \Phi$. Each $-1<y<1$ has two preimages in one revolution. The change of variables formula adds both contributions:The affine change of variables $X=(1+Y)/2$ gives the standard arcsine distribution. A single monotone-branch calculation without the second preimage would give only half the total probability.

The same geometry explains the ideal harmonic oscillator $y(t)=A\cos (\omega t+\varphi )$, with amplitude $A>0$ and angular frequency $\omega >0$. Choose $T$ uniformly in any complete period of duration $2\pi /\omega$. Its oscillation phase modulo $2\pi$ is uniform, and $Y=y(T)$ has densityNear a turning point, $|dy/dt|=\omega \sqrt{A^2-y^2}$ is small, so a fixed position interval takes more time to cross. For the chosen amplitude $A=2\mathrm{cm}$, exactly one third of the period is spent in $|y|\le 1\mathrm{cm}$, and two thirds outside that central interval. These are calculations for an ideal sinusoid, rather than measured oscillator data.

The sampling assumption matters. Taking one observation every full period returns the same position each time. It does not produce this probability distribution. Uniformly random observation times and a fixed regularly spaced sampling scheme are different models; aliasing can conceal the motion altogether.

Paul Lévy used a related geometric construction in his work on stochastic processes: project a uniformly chosen point on a circle onto its diameter to divide a segment. The projection description appears explicitly in §5 of his paper, following its arcsine density calculation.^[ref]

Specify whether two parameters mean endpoints or location and width. With endpoints $a<b$, define $Y=a+(b-a)X$ for a standard arcsine distribution variable $X$. ThenThis is a location-scale family with location $a$ and width $b-a$. Its mean is $(a+b)/2$ and its variance is $(b-a{)}^2/8$. A convention using parameters $a,w$ for location and width instead has right endpoint $a+w$, so it must not be read as having endpoints $a,w$.^[ref]

For example, a chosen interval $[-2,6]$ gives mean $2$ and variance $8$, while its density still concentrates toward both endpoints. It is a bounded probability distribution; its endpoint singularities do not make its support infinite.

Inverting $F(x)=p$ gives the quantile functionConsequently, inverse transform sampling uses $X={\sin }^2(\pi U/2)$ with $U$ uniform on $(0,1)$. The quartiles are $(2-\sqrt{2})/4$, $1/2$ and $(2+\sqrt{2})/4$.^[ref]

The first and third quartiles are about $0.1464$ and $0.8536$, so the central half of the probability spans about $0.7071$ of the unit interval. Under a uniform distribution, its width would be $0.5$.

The symmetric probability distribution gives $\mathbb{E}[X]=1/2$. The angle representation gives a short calculation of the second moment:Use ${\sin }^4\theta =(3-4\cos 2\theta +\cos 4\theta )/8$ to evaluate the integral. A uniform distribution on $[0,1]$ has variance $1/12$; the larger variance $1/8$ quantifies the greater spread toward the ends.

More generally, for a nonnegative integer $n$, the beta function yieldsFor $n=0$ this is one; for $n=1,2$ it reproduces the values above. The central binomial coefficient thus appears in an exact moment calculation for a continuous probability distribution.

Let $Z$ have the fair Bernoulli distribution and set $X_j=Z$ for every $j$. Each coordinate has the same fair probability distribution as in the independent model, but the coordinates are dependent: $\mathbb{P}(X_1=1,X_2=1)=1/2$, whereas the product of the marginal probabilities is $1/4$. Every future sigma-algebra equals $\sigma (Z)$, so the tail sigma-algebra also equals $\sigma (Z)$.

The tail event of infinitely many heads is now exactly $\{Z=1\}$, with probability $1/2$. For the finite-window events $B_{m,r}$ in infinitely many heads as a tail event,The sample space can even be the same full Cartesian product $\{0,1{\}}^{\mathbb{N}}$ in both models: the dependent probability measure assigns mass $1/2$ to each of the all-zero and all-one sequences. Then $A$ is the same literal tail event in both models. Only the probability measure and its independence properties change.

For the MA(1) $X_t=\mu +{\epsilon }_t+\theta {\epsilon }_{t-1}$, centered white noise givesOnly products of noise values with matching time indices have nonzero expectation. At lag zero there are two squared weights, $1$ and ${\theta }^2$; at lag one there is one shared noise term with weight product $\theta$; at higher lags there are none. The calculation needs zero covariance between distinct noise values, not independence. This tridiagonal finite covariance structure makes the finite-sample innovations of an MA(1) process particularly simple.

The lag-one autocorrelation can never exceed $1/2$ in magnitude, since $1+{\theta }^2\ge 2|\theta |$. For $\theta \ne 0$, replacing $\theta$ by $1/\theta$ leaves the entire ACF unchanged. For example $\theta =2$ and $\theta =1/2$ both give $\rho (1)=2/5$; rescaling the latter noise variance by $4$ also matches every autocovariance. Moving-average root reflection explains this ambiguity and why a stable invertibility convention helps identify a representation. Equality of second-order laws need not imply equality of all joint probability distributions unless the processes are Gaussian processes. See Bartlett, lecture 6: equivalent representations and covariance of ARMA processes.

For the identity design and $y={\theta }^{\ast }+z$, comparing the minimizing objective at $\hat{\theta }$ with its value at ${\theta }^{\ast }$ and expanding the squared Euclidean norm proves the displayed inequality, with $h=\hat{\theta }-{\theta }^{\ast }$. This deterministic inequality does not assume any probability distribution for the noise. A bound on the noise inner product turns it into a statistical error estimate.

Bayesian statistics expresses uncertainty about an unknown statistical parameter through a probability distribution, then uses Bayes' theorem to update it after observing data. The prior distribution describes uncertainty before the current observations; the likelihood function describes how the statistical model generates those observations at each parameter value; the posterior distribution describes uncertainty conditional on the observations and that model. A physical parameter can have one fixed value while its value remains uncertain to us. Assigning it a probability distribution represents that uncertainty under the chosen model, rather than requiring the physical parameter to change randomly between measurements.

For example, suppose ten Bernoulli trials are independent conditional on an unknown heads probability $\theta$, with a beta distribution prior $\mathrm{Beta}(2,2)$. Eight heads and two tails give the posterior distribution $\mathrm{Beta}(10,4)$ by Beta-binomial conjugacy. Its posterior mean is $10/14=5/7\approx 0.714286$, between the prior mean $1/2$ and observed binomial proportion $8/10$. The Bayesian posterior below computes the whole update, and the credible interval below summarizes its remaining uncertainty. The conjugate update is explained in MIT 18.05, Reading 15: beta and normal conjugate priors.

The useful feature is that one posterior distribution supports several questions: a posterior mean estimates a value, a credible interval reports a range, and a posterior predictive distribution predicts a future observation while averaging over parameter uncertainty. With conditionally independent new data, today's posterior distribution can become tomorrow's prior distribution. These conclusions remain conditional on the statistical model and prior distribution; a precise update does not establish that either assumption fits the experiment.

A historical starting point is Thomas Bayes's essay in the 1763 volume of the Royal Society's Philosophical Transactions, communicated after his death by Richard Price. Its opening problem asks for uncertainty about an unknown event probability given observed successes and failures: the same direction of reasoning illustrated by the coin example. See Bayes and Price, An Essay towards solving a Problem in the Doctrine of Chances, opening letter and problem.

An assignment of an action probability distribution to every information set of a player in an extensive-form game. Randomization is fresh at each encounter. It differs from a mixed strategy, which chooses a complete deterministic plan once before play. Perfect recall is central to equivalence of their induced outcome distributions.

Without a prior probability distribution, a belief state is the set of physical states still compatible with an observed history. If the physical search transition model is deterministic, $T(s,a)$, and the sensor map is $O$, action $a$ followed by observation $o$ givesFor a set-valued outcome relation, replace the image by the set union of all possible successors before filtering. A guaranteed-success goal test requires every member of the surviving belief state to satisfy the physical goal. A probabilistic belief is instead a posterior probability distribution, requiring a specified prior and observation law.

Different precise measures on random chords give different probabilities; the phrase “random chord” does not select a unique probability distribution. In a circle of radius $r$, a chord exceeds the side of an inscribed equilateral triangle exactly when its midpoint distance from the center is below $r/2$. Uniform midpoint area gives probability $1/4$, independent uniform endpoints give $1/3$, and uniform midpoint distance gives $1/2$. The difference is in the sampling laws, not a contradiction in probability theory.

For probability distributions $p,q$ on the same finite alphabet, the Bhattacharyya coefficient measures their square-root overlap:The vectors $(\sqrt{p_i}{)}_i$ and $(\sqrt{q_i}{)}_i$ have unit Euclidean norm, so the Cauchy-Schwarz inequality gives $0\le B\le 1$. Equality $B=1$ holds exactly when the distributions agree; $B=0$ holds exactly when they assign positive probability to disjoint outcomes. For $p=(0.9,0.1)$ and $q=(0.6,0.4)$, $B=\sqrt{0.54}+0.2\approx 0.934847$. This overlap is not the probability that two independent draws agree, which here is $0.58$.

The fidelity of commuting mixed states reduces to this coefficient. More generally, the measurement characterization of quantum fidelity minimizes it over outcome distributions of a common quantum measurement, connecting classical overlap to the unsquared quantum fidelity.

The Bhattacharyya distance is the negative natural logarithm of the Bhattacharyya coefficient, with $-\ln 0=\infty$. It is zero for identical probability distributions and infinite for disjoint probability supports. Despite the name, it need not satisfy the triangle inequality required of a metric.

A binary symmetric channel independently flips each transmitted bit with crossover probability $p$. Its channel capacity isin bits per use, where $h_2$ is binary entropy and the endpoint convention is $0{\log }_{2}0=0$. Knowing the input leaves just the uncertainty of the flip, so $H(Y\mid X)=h_2(p)$ for every input probability distribution. Since the output is a bit, $H(Y)\le 1$, and thus $I(X;Y)\le 1-h_2(p)$. A fair input produces a fair output and reaches the bound. This is the two-symbol case of weakly symmetric channel capacity; Eytan Modiano, MIT 16.36, Channel Capacity and Coding, slides 4–5 gives this calculation.

For crossover probabilities $p=0.01,0.1,0.25$, the resulting channel capacities are approximately $0.91920686$, $0.53100441$, and $0.18872188$ bits per use. At $p=0$, the communication channel is noiseless; at $p=1/2$, the output is independent of the input and the capacity is zero. At $p=1$, every bit is inverted, a known reversible rule, and capacity is one again. More generally $C(p)=C(1-p)$, because inverting every received bit exchanges the two crossover probabilities. The existing mutual information figure shows this full symmetric curve.

An uncoded transmission illustrates why the raw bit-error probability is not itself channel capacity. At $p=0.1$, transmitting 100 uncoded bits succeeds on all of them with probability $0.9^{100}\approx 0.00002656$. The probability of at least one error is therefore approximately $0.99997344$. A suitable block code changes which received words count as recoverable evidence of a message; simply repeating uncoded transmission at rate one does not obtain the reliability promised at rates below $C$.