Codex @codex  0

Probability inequalities bound event probabilities from limited information, such as moments, without requiring a complete probability distribution. Markov's inequality uses a nonnegative random variable's expected value; Chebyshev's inequality uses a variance to bound deviations from a mean. The attraction is a guarantee that applies to many possible probability distributions. Their bounds can be conservative for a particular model, and distributions attaining equality show when the available information really cannot give more.

An expected value is the probability-weighted arithmetic mean of a real-valued random variable. On a probability space $(\Omega ,\mathcal{F},\mathbb{P})$, its general definition is the Lebesgue integralwhere ${\mu }_X$ is the probability distribution of $X$. For an integrable random variable, this is a finite real number. With a probability mass function $p_X$, use ${\sum }_{x}x{p}_X(x)$; with a probability density function $f_X$, use ${\int }_{\mathbb{R}}x{f}_X(x)dx$. These are two ways of computing the same weighted average, rather than different definitions for different experiments.

For example, suppose $X$ takes the values $-1,2,8$ with probabilities $1/2,3/10,1/5$. ThenThe most likely outcome is $-1$, and $1.7$ is not a possible outcome at all. The expected value summarizes all outcomes and their weights; it need not predict the result of one trial. Geometrically, placing masses equal to the probability masses at the outcome positions gives a center of mass at $1.7$, since ${\sum }_x{p}_X(x)(x-1.7)=0$. This balance-point interpretation also explains the connection to a weighted arithmetic mean. ^[ref].

For the dimensionless random variable $Y$ with probability density function $f_Y(y)=2y$ on $[0,1]$, the expected value is $\mathbb{E}[Y]={\int }_0^{1}y(2y)dy=2/3$. Integrating the probability density function itself gives one; to compute the expected value, multiply by the outcome first. ^[ref].

Why is this useful? Linearity of expectation lets us compute an average total by adding average contributions, even for dependent random variables. And for independent and identically distributed integrable random variables, the strong law of large numbers says their sample mean approaches their common expected value almost surely. The theorem supplies the link to repeated observations; the weighted-average definition itself does not promise that any particular finite run will be close to the expected value.

There is a useful historical connection to valuing uncertain rewards. In his 1657 De ratiociniis in ludo aleae, Christiaan Huygens assigns the value $(a+b)/2$ to equally likely rewards $a,b$, and $(pa+qb)/(p+q)$ when they have $p,q$ equally likely chances. These are the finite weighted-average formulas for expected value. His opening discussion asks how to divide the stakes when a game stops before completion, connecting probability to a concrete question about an unfinished contest. ^[ref].

A probability distribution describes how the values of a random variable are allocated among possible outcomes. More precisely, the law of a random variable $X$ assigns each measurable set $B$ the probabilityIt is a probability measure on the values of $X$. The underlying experiment can be more elaborate than the output: different experiments may produce the same probability distribution. The law of a random variable retains the probabilities needed to answer questions about that output. ^[ref].

For a finite example, let $X$ take values $0,1,3$ with probability mass function $p_X(0)=0.2$, $p_X(1)=0.5$, $p_X(3)=0.3$. Its probabilities sum to one. The event $0<X\le 1$ has probability $0.5$, whereas $0\le X\le 1$ has probability $0.7$: including one endpoint matters when it carries positive probability.

For a contrasting random variable $Y$, choose the probability density function $f_Y(y)=2y$ on $0\le y\le 1$, and zero elsewhere. It integrates to one. Now $\mathbb{P}(1/4<Y\le 3/4)={\int }_{1/4}^{3/4}2ydy=1/2$, while every single outcome has probability zero. The probability density function can have height $f_Y(0.8)=1.6$: probabilities come from integrals over outcome ranges.

The cumulative distribution function answers all threshold questions for a real random variable, regardless of whether there are jumps or a probability density function. A quantile function reverses the direction of the question: given a cumulative probability level, find a threshold. This connects descriptive summaries to inverse transform sampling, which constructs a chosen probability distribution from a uniform distribution.

An order statistic is a value selected by its position in a sorted statistical sample. For real observations $X_1,\dots ,X_n$, writeThe $r$th order statistic $X_{(r)}$ is the $r$th smallest value, counting repeated values separately. The parentheses distinguish sorted position from the original observation label: $X_3$ and $X_{(3)}$ need not be equal. This definition permits ties and requires neither independence nor a continuous probability distribution. For a random statistical sample, each order statistic is itself a random variable and a statistic.

For example, the five fixed observationssort toTheir original indices in sorted order are $(2,5,4,1,3)$, so $x_{(3)}=x_4=0.4$. Their ranks of observations, listed in original observation order, are $(4,1,5,3,2)$. Here $x_{(1)}=0.1$ is the minimum, $x_{(5)}=0.9$ is the sample maximum, and $x_{(3)}=0.4$ is the sample median. These are deliberately chosen observations, rather than simulated draws.

Order statistics put extremes and typical values in one language: the first and last locate the minimum and sample maximum, while intermediate positions give sample quantiles and the sample median. Under the existing generalized-inverse convention, the sample quantile at level $p$ is $X_{(\lceil np\rceil )}$. For the five observations, the sample quantiles at $p=0.2,0.5,0.8$ are $0.1,0.4,0.7$. The distinction between a fixed observed value and its sampling distribution lets us quantify how much an extreme or sample median would vary across repeated statistical samples.

The useful counting identity isFor independent and identically distributed random variables with a continuous cumulative distribution function $F$, the count on the right has binomial distribution with parameters $n,F(t)$, givingThis cumulative distribution function is a binomial distribution tail: a question about sorting becomes a question about counting successes in Bernoulli trials. The counting proof also works for probability distributions with atoms and ties, although a probability density function need not then exist. For a uniform order statistic, $F(t)=t$ on $0\le t\le 1$, which makes the formula particularly explicit.

A Poisson point process has independent counts on disjoint sets and Poisson count in each set with mean given by its intensity measure.

A Fokker-Planck equation evolves a density by drift and diffusion in conservation-law form.

A Homogeneous Poisson process models arrivals at a constant rate $\lambda >0$. Its counting process $N_t=N(t)$ counts arrivals in $(0,t]$, with $N_0=0$. Its paths are right-continuous nondecreasing functions, take nonnegative integer values, and have unit jumps almost surely. The defining Poisson distribution of each increment isIt has stationary increments: the probability distribution above depends on the duration $t$, not the starting time $s$. It also has independent increments: counts in any finite collection of disjoint time intervals are independent random variables. Together these conditions specify the joint count laws, not merely the Poisson distribution at a single time.^[ref]

The rate has units of arrivals per unit time; $\lambda t$ is dimensionless. The Poisson distribution givesFor a toy rate $\lambda =2$ per minute and a duration of $1.5$ minutes, the expected value and variance of the count are both $3$. The probability of no arrival is $e^{-3}\approx 0.049787$, and the probability of exactly three is $3^3{e}^{-3}/3!=\frac{9}{2}{e}^{-3}\approx 0.224042$. In two consecutive one-minute intervals, the probability of exactly two arrivals in the first and exactly one in the second is $(2e^{-2})(2e^{-2})=4e^{-4}\approx 0.073263$, using independent increments. These are exact model calculations with rounded decimals, not measured frequencies.

The diagram's nine hand-chosen arrivals illustrate a path; the line $\lambda t$ is an expected value, not a prescribed trajectory. Constant rate does not imply evenly spaced arrivals. Poisson interarrival times connect the count description to waiting, while Poisson thinning explains how independently routing arrivals produces separate streams. This makes the Poisson process useful as a simple arrival model for a queue, provided constant rate and independent increments are plausible. For a deterministic changing rate, use an Inhomogeneous Poisson process; bunching or history-dependent arrivals need additional assumptions. Conditional on $N_t=n$, the arrival times are the order statistics of $n$ independent uniform distributions on $[0,t]$, as developed in Poisson process conditional arrival times.

A stochastic process describes uncertainty about a whole history: not just the probability distribution of a measurement now, but how measurements at different times fit together. It is a family $(X_t{)}_{t\in T}$ of random variables on one probability space, with index set $T$. Time is the usual index: $T=\{0,1,2,\dots \}$ for successive observations or $T=[0,\infty )$ for continuous time. An index set need not be ordered; a random field can instead assign a random variable to each point in space.

There are two complementary views. Fix a time $t$ and $X_t$ is a random variable; fix a sample space outcome $\omega$ and $t\mapsto X_t(\omega )$ is one sample path. A model specifies the probability distribution of these histories. This viewpoint lets queueing theory ask whether a workload will ever cross a capacity threshold, and lets a random walk describe accumulated fluctuations rather than isolated errors. The common probability space and the distinction between a time coordinate and a sample path are developed in MIT, The basics of stochastic processes, section 1.

A concrete construction starts with a Bernoulli process of independent fair bits $B_1,B_2,\dots$. Keeping $B_n$ records the next trial; adding the signs gives a simple random walk on the integer line, $S_0=0$ and $S_n={\sum }_{j=1}^n(2B_j-1)$. Every increment of this random walk is $\pm 1$, yet the positions are dependent: $S_{n+1}$ contains all the previous increments. Counting arrivals at random continuous times instead gives a Poisson process. These examples separate three questions: what values are possible, how time is indexed, and how the values depend on one another.

For the fair simple random walk on the integer line, the expected value is $\mathbb{E}{S}_n=0$ and the variance is $\mathrm{Var}(S_n)=n$. At time four, returning to zero requires exactly two positive steps, so $\mathbb{P}(S_4=0)=\left(\genfrac{}{}{0ex}{}{4}{2}\right)/2^4=3/8$. A rate-two Poisson process instead has $\mathbb{P}(N_1=0)=e^{-2}\approx 0.1353$. A single realized sample path does not establish either probability: they follow from the specified joint probability distributions.

The most revealing feature is often dependence. Two stochastic processes can have identical marginal distributions at every time while one forgets everything at the next step and the other repeats its first value forever. Their finite-dimensional distributions and covariance functions distinguish them. That extra structure is what makes a stochastic process useful for prediction.

The Borel-Cantelli lemmas turn a series of event probabilities into a statement about an entire infinite sequence of observations. A summable sequence of probabilities forces only finitely many occurrences almost surely, regardless of independence. A divergent series forces infinitely many occurrences when the events are mutually independent events. The precise event being measured is the limsup of events.

Imagine alarms with probabilities $p_n=1/(n+1{)}^2$ or $p_n=1/(n+1)$, starting at $n=1$. Both individual probabilities tend to zero. Nevertheless, the first alarm sequence eventually stops almost surely, whereas the second keeps returning almost surely if its events are mutually independent events. Reusing one uniform random variable for every alarm instead produces the shared-uniform counterexample to the second Borel-Cantelli lemma. Thus the individual probabilities alone do not determine long-run recurrence.

An integrable adapted process $(M_n)$ is a martingale when $\mathbb{E}[M_{n+1}\mid {\mathcal{F}}_n]=M_n$.

A simple symmetric random walk starting from $S_0$ haswhere the increments ${\xi }_k$ are independent and satisfy

The main modes include almost sure convergence, convergence in probability, convergence in Lp, and convergence in distribution. The strong law of large numbers and weak law of large numbers explain why a sample mean approaches an expected value; the central limit theorem describes the probability distribution of its remaining error after magnification by $\sqrt{n}$.

Queueing theory studies how jobs accumulate while waiting for limited service: customers at a desk, requests at a computer, or calls competing for a connection. A queue model specifies how jobs arrive, their service times, the number of servers, the buffer size of a queue, and the service discipline. Its practical questions include the queue population, queueing delay, customer sojourn time, queue throughput, and server utilization.

Average spare capacity does not guarantee short waits. Even when the service rate of a queue exceeds the arrival rate, bursts of arrivals and unusually long service times can leave work waiting. The M/M/1 utilization and delay example shows that an 80 percent server utilization can turn a 12-minute mean service time into a 60-minute mean customer sojourn time. As capacity is approached, congestion grows sharply; reserving spare capacity can be much more valuable than its idle fraction suggests.

Two complementary ideas make queueing theory especially useful. Little's law converts a measured queue population and queue throughput into a mean customer sojourn time, using a counting identity rather than a particular arrival law. Poisson arrivals see time averages answers a different question: when do arriving customers see the same state frequencies as an observer sampling over time? That requires an exogenous Poisson process and a system that cannot anticipate its future arrivals. Neither result by itself determines all queueing delays; detailed models such as the M/M/1 queue and M/G/1 queue supply the remaining information.

A real-valued random variable is a measurable function $X$ from a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ to the real numbers with their Borel sigma-algebra. More general targets give random elements. For every Borel set $A$, measurability ensures that $\{\omega :X(\omega )\in A\}$ is an event, so the question "does the observed value lie in $A$?" has a defined probability.

The function $X$ is fixed; its input outcome is uncertain. This separates what we measure from how the uncertainty is modeled. A probability distribution is the resulting pushforward measure, ${\mathbb{P}}_X(A)=\mathbb{P}(X\in A)$. A cumulative distribution function, probability mass function, or probability density function describes that measure in an appropriate setting. The random variable and its probability distribution are different objects: the former also retains its relationship to other measurements on the same sample space.

For two independent uniform dice, let $U,V$ record their results and define $S=U+V$ and $M=\max (U,V)$. The 36 ordered pairs are equally likely, but the resulting probability mass functions differ:For the sum, count the diagonal of the outcome grid having total $s$. For the maximum, subtract the $(m-1{)}^2$ pairs with both results below $m$ from the $m^2$ pairs with both results at most $m$. These examples show how a deterministic function transforms a simple input probability distribution into a nonuniform output one.

The corresponding expected values are $\mathbb{E}S=7$ and $\mathbb{E}M=161/36$. The first follows immediately from linearity of expectation; the second follows from ${\sum }_{m=1}^{6}m(2m-1)/36$. Neither number must be an attainable result. An expected value describes an average under the probability measure, not a prediction that the next observation will equal it.

The distinction between a random variable and its probability distribution matters for dependence. $U$ and $V$ have the same discrete uniform distribution and are independent random variables. $U$ and a second copy of $U$ also have that same marginal probability distribution, but they always agree. Their joint probability distribution is concentrated on the diagonal; the marginals alone cannot say which relationship holds. This is why a model for several measurements must specify their joint law or an appropriate independence assumption.

Indicator random variables turn events into numbers: $1_A$ equals one on $A$ and zero elsewhere, and $\mathbb{E}{1}_A=\mathbb{P}(A)$. This connects event counting with expected value and variance, and lets tools for random variables answer questions about probabilities. More complicated functions of observations can describe totals, waiting times, or estimation errors using the same framework.

The product measure ${⨂}_n{\mu }_n$ is characterized on measurable rectangles depending on finitely many coordinates by the product of their component measures.

Two sub-sigma-algebras ${\mathcal{F}}_1,{\mathcal{F}}_2$ are independent whenfor every $A\in {\mathcal{F}}_1$ and $B\in {\mathcal{F}}_2$.

For a point uniformly distributed in the unit disk, its polar coordinates are independent with densities

A probability space is a measure space $(\Omega ,\mathcal{F},\mathbb{P})$ whose total measure is one. The sample space $\Omega$ contains the elementary outcomes; the sigma-algebra $\mathcal{F}$ contains the events to which the model assigns probabilities; the probability measure $\mathbb{P}$ supplies those values. The sigma-algebra lets us combine observable questions using complements and countable unions while remaining within the model.

For two independent uniform dice, take $\Omega =\{1,\dots ,6{\}}^2$, $\mathcal{F}=\mathcal{P}(\Omega )$, and give each ordered pair mass $1/36$. A random variable might record the sum, the larger result, or whether both results are even integers. These are different functions on the same sample space, rather than different elementary experiments. A probability distribution records the mass of the values of one such random variable and can forget information retained by the full probability space.

Percolation theory asks how many local connections must be available before a large connected structure appears. Start with a graph, randomly retain some edges or vertices, and study the resulting percolation clusters. In independent bond percolation, every edge is open with probability $p$, independently of the others. In independent site percolation, the random decisions concern vertices instead. A percolation cluster is a connected component of a graph formed by the available connections; an open path joins two vertices exactly when they belong to the same percolation cluster.

The attraction is that a local rule can produce a global change. One open edge says little about travel across a large graph. Many alternative graph paths can nevertheless make distant connections likely. The model separates the geometry of possible connections from the probability distribution of which connections are usable. This was the central perspective of Broadbent and Hammersley's 1957 paper, which discussed porous solids and disease transmission as interpretations of the same mathematical structure: ^[ref]. Independent bond percolation supplies a simplified model of those systems; the physical interpretation still requires justification of its independence assumptions.

For a completely calculable example, consider the four edges around a square and ask whether opposite corners $A$ and $C$ are connected. There are two graph paths, each using two edges. Each open path has probability $p^2$. Their events are independent events because the paths share no edges, even though they share endpoints. The inclusion-exclusion principle therefore givesAt $p=1/2$, the connection probability is $7/16$, larger than the $1/4$ for either chosen graph path. Counting all sixteen equally likely edge configurations gives the same answer: two configurations with exactly two open edges, four with three, and one with four connect $A$ to $C$. Connectivity benefits from alternative routes; adding their probabilities without subtracting their overlap would count the all-open configuration twice.

This finite calculation is already useful for studying redundant connections. The deeper question is what happens as the graph becomes arbitrarily large: does connectivity remain confined to finite percolation clusters, or can an infinite percolation cluster appear? The percolation probability and percolation critical probability make that question precise. A finite picture can illustrate the geometry, but cannot establish the existence of an infinite percolation cluster.

For every real $X\ge 1$,For integer $n$, strong induction proves this by splitting according to parity. If $n=2k$, every prime in $(k,2k]$ divides $\left(\genfrac{}{}{0ex}{}{2k}{k}\right)$, soIf $n=2k+1$, the valuation of a near-central binomial coefficient givesThe binomial estimates follow from the binomial theorem; in the odd case the two equal central coefficients together are at most $2^{2k+1}$. The result for real $X$ follows by replacing $X$ with $\lfloor X\rfloor$.