Stochastic process Created 2026-09-24 Updated 2026-10-14
A stochastic process describes uncertainty about a whole history: not just the probability distribution of a measurement now, but how measurements at different times fit together. It is a family of random variables on one probability space, with index set . Time is the usual index: for successive observations or for continuous time. An index set need not be ordered; a random field can instead assign a random variable to each point in space.
There are two complementary views. Fix a time and is a random variable; fix a sample space outcome and is one sample path. A model specifies the probability distribution of these histories. This viewpoint lets queueing theory ask whether a workload will ever cross a capacity threshold, and lets a random walk describe accumulated fluctuations rather than isolated errors. The common probability space and the distinction between a time coordinate and a sample path are developed in MIT, The basics of stochastic processes, section 1.
A concrete construction starts with a Bernoulli process of independent fair bits . Keeping records the next trial; adding the signs gives a simple random walk on the integer line, and . Every increment of this random walk is , yet the positions are dependent: contains all the previous increments. Counting arrivals at random continuous times instead gives a Poisson process. These examples separate three questions: what values are possible, how time is indexed, and how the values depend on one another.
Figure 1.
Three kinds of random history: fair bits, their accumulated signs, and Poisson arrival counts
. Details.
For the fair simple random walk on the integer line, the expected value is and the variance is . At time four, returning to zero requires exactly two positive steps, so . A rate-two Poisson process instead has . A single realized sample path does not establish either probability: they follow from the specified joint probability distributions.
The most revealing feature is often dependence. Two stochastic processes can have identical marginal distributions at every time while one forgets everything at the next step and the other repeats its first value forever. Their finite-dimensional distributions and covariance functions distinguish them. That extra structure is what makes a stochastic process useful for prediction.
Stopping time Created 2026-09-24 Updated 2026-11-05
Relative to a filtration , a stopping time is a random variable with values in such that
In discrete time, use integer-valued and integers . The event that stopping has occurred must be decidable from the information available now. The stopping-time sigma-algebra records the information available at .
For a continuous adapted process, its first hit of a closed upper level is a stopping time, as shown by closed-level hitting times of continuous adapted processes. A fixed deterministic time also qualifies. The time of the maximum of a Brownian motion over a prescribed future interval generally does not: deciding whether the final maximum has already occurred requires future observations. This distinction determines whether the Strong Markov property or an optional stopping theorem applies.
Strict positive covariance with an increasing payoff Created 2026-10-06 Updated 2026-11-05
Let be a nondegenerate real random variable and a strictly increasing function. Assume , and are integrable. For an independent copy ,
The product is positive whenever and zero otherwise. Nondegeneracy gives positive probability of unequal copies, proving strict positivity. The same argument applies to a conditional distribution when that conditional law is nondegenerate and has the required integrability.
For a market with one trading period, suppose an increasing terminal payoff is replicated by a risk-free asset holding and a stock holding: . Under current information , let be -measurable and . Then
This is a conditional one-period statement. General dynamic attainability alone does not specify the affine representation used in this calculation.
Subadditivity of information entropy Created 2026-09-24 Updated 2026-11-05
For discrete random variables , Shannon entropy satisfies
When the two marginal Shannon entropies are finite, the chain rule for information entropy and conditioning reduces entropy prove the inequality, and the difference is the mutual information . Thus equality holds exactly when and are independent random variables.
For example, let be a fair Bernoulli random variable. Then bit, while bits: the pair does not contain two independent copies of the information. The inequality also holds with extended nonnegative values, but equality of two infinite values does not characterize independence.
Subexponential distribution (light-tailed) Created 2026-10-13 Updated 2026-11-05
A centered random variable is sub-exponential with parameters if
where . Unlike a sub-Gaussian random variable, the moment-generating-function bound is needed only near zero. Optimizing exponential Markov inequality over this restricted interval gives a two-sided tail bound of the form
with an absolute constant . Small deviations therefore have a quadratic exponent and large deviations a linear exponent. Equivalently, up to constant changes in parameters, is bounded for some finite scale .
For an exponential random variable of mean one, has moment-generating function for . Its logarithm is , which is at most for , proving a sub-exponential bound. The Centered square of a sub-Gaussian random variable is another example. The term here refers to exponential-tail concentration, and differs from the subexponential distribution defined by heavy-tail convolution asymptotics.
Sub-Gamma random variable in the right tail Created 2026-09-24 Updated 2026-11-05
A centered random variable is sub-Gamma in the right tail with variance parameter and scale parameter when
for .
A sub-Gaussian deviation bound controls tails through a Gaussian-shaped exponential. If a centered random variable satisfies for every real , with , then for ,
Indeed, exponential Markov's inequality gives ; minimizing at and applying the argument to proves the result.
Independent centered observations with a common variance proxy therefore satisfy . The proxy need not equal the actual variance, and dependence requires a separate argument rather than automatic multiplication of moment-generating functions.
Sub-Gaussian distribution Created 2026-10-13 Updated 2026-11-05
A real centered random variable is sub-Gaussian random variable with parameter if its moment-generating function satisfies
The parameter is an upper bound on fluctuation size, not necessarily the standard deviation. For and , exponential Markov inequality gives . Choosing and applying the same argument to yields the Chernoff bound
A centered normal distribution of variance attains equality in the moment-generating-function condition. A Rademacher distribution also satisfies it with , since . Products of moment-generating functions show that a sum of independent centered sub-Gaussian random variables with parameters is sub-Gaussian with parameter . Hence the sample mean of independent variables with a common parameter has tail bound . This gives normal-like concentration without assuming a normal distribution. For an uncentered variable, the convention is to apply the condition to .
Sub-Poisson random variable in the right tail Created 2026-09-24 Updated 2026-11-05
A centered random variable is sub-Poisson in the right tail with variance parameter when
for every .
Super-uniform random variable Created 2026-09-24 Updated 2026-11-05
A random variable is super-uniform when for every . A valid p-value is super-uniform under its null hypothesis.
If a finite group acts by measure-preserving maps on a probability space, let be its invariant sigma-algebra. For an integrable random variable , the displayed group average is -measurable. For , changing variables under each group element gives . The average therefore satisfies the defining integral identity for conditional expectation. For permutations of exchangeable coordinates, averaging a product attached to distinct indices yields the sum over all ordered distinct -tuples divided by the falling factorial .
Tail event Created 2026-09-24 Updated 2026-10-14
A tail event is an event , where is the tail sigma-algebra of the chosen sequence of random variables. It concerns information available arbitrarily far along the sequence. In the measurable coordinate models described in finite-prefix invariance of tail events, any finite edit preserves whether it occurs.
This is stronger than merely needing a long waiting time. On , with denoting heads, let , taking if there is no head. The event
is not a tail event: changing only from zero to one changes membership. Under fair independent Bernoulli trials its probability is . Even the event , of probability one in this model, is not a tail event as a literal set: the all-zero sequence and the sequence with just one initial head differ in one coordinate. A probability of zero or one does not characterize tail events.
Tail integral formula for expectation Created 2026-10-14 Updated 2026-10-20
For a nonnegative random variable, pointwise, including when . Tonelli's theorem permits taking expectation under the integral and proves the formula in the extended nonnegative reals. This relates finiteness of a Brownian exit time expectation to integrability of its survival probability.
Tail integral formula for moments Created 2026-09-24 Updated 2026-11-05
For and a real random variable , the tail integral formula for moments is
valid even if both sides are infinite. Pointwise, ; Tonelli theorem interchanges this nonnegative integral with expectation. Replacing by does not change the integral, since their difference is confined to the at most countably many atoms of the distribution. For an exponential random variable of rate , the formula at gives .
Tail measurability of limits of sample averages Created 2026-10-05 Updated 2026-10-14
Let be a sequence of finite real random variables, and write for its sample mean. For fixed and , put . Then, at each outcome,
Consequently . The right-hand limit superior is a measurable function of for every , so the extended-real random variable is measurable with respect to the tail sigma-algebra. The same holds for the limit inferior. Thus the event of convergence to a finite limit of a sequence, and the event for any specified real , are tail events.
For a worked deterministic example, let alternate , and replace its first three entries by . For every the two sample means differ by exactly , so both converge to . Finite edits can change small-sample answers while preserving the infinite limit of a sequence.
Under independence, Kolmogorov's zero–one law makes each specified convergence event have probability zero or one. It does not select which value occurs. If a finite random variable measurable with respect to the tail sigma-algebra exists, the events for rational each have probability zero or one; continuity of probability and their nesting force to be constant almost surely. For fair independent Bernoulli trials, the strong law of large numbers establishes almost surely; the zero–one conclusion alone proves neither convergence nor the value .
Tail probability Created 2026-10-03 Updated 2026-10-14
For a real random variable , an upper-tail probability has the form and a lower-tail probability has the form . Each is the probability of an event lying beyond a chosen threshold in one direction. A tail probability need not be small: the threshold determines how much of the probability distribution it includes. A concentration inequality bounds such probabilities, often for deviations from an expected value.
Endpoint conventions matter. Write for the cumulative distribution function and . The complement rule of probability gives four distinct formulas:
The strict upper-tail convention is the survival function . Its value differs from the inclusive upper tail probability by exactly the probability mass at the threshold:
A singleton with positive probability mass is an atom of a measure. Thus excludes an atom of a measure at , whereas includes it. These endpoint identities follow directly from the cumulative distribution function properties in Kyle Siegrist, Distribution and Quantile Functions: endpoint probabilities and right tails.
As the threshold increases, both upper tail probabilities are nonincreasing functions. The strict survival function is a right-continuous function; the inclusive upper tail probability takes the upper endpoint value at a downward jump, equal to the left limit of the survival function. Both tend to zero as for a real random variable, but the speed of that decay differs across probability distributions. The cumulative distribution function properties and their proof from continuity of probability measures are given in MIT 6.436J, Lecture 4: random variables and cumulative distribution function properties.
For a fair six-sided die , the discrete uniform distribution assigns probability mass to each of . At threshold four,
Between four and five, both upper tail probabilities equal . At the largest possible outcome the distinction is especially visible: , but . The diagram shows exact probabilities for this mathematical model; its dots specify which endpoints belong to each step function.
Figure 1.
Exact fair-die probabilities: including or excluding a threshold changes the tail by one atom
. Details.
If a probability distribution has a probability density function , then every singleton has zero probability mass and
For an exponential distribution of rate two, for and zero otherwise, so for and for . Hence . This tail probability is an integral of the probability density function, not its height . A mixed probability distribution retains endpoint jumps even with a smooth component: give zero probability mass and choose this exponential distribution with probability . Then , , and for .
A two-sided tail probability can measure the absolute value of displacement from a specified center . For , the two extreme events are disjoint events, so
For strict displacement , replace the last expression by . The condition prevents overlap; at adding inclusive tails double-counts any probability mass at . For a uniform distribution on , take and : the two tail probabilities are each and their sum is . Equal tails follow from this symmetric probability distribution, not from the phrase two-sided alone; for a uniform distribution on with , the two tail probabilities are zero and .
The tail-sum formula for expectation turns a whole curve of tail probabilities into an expected value. For any nonnegative random variable , without assuming a probability density function or a finite expected value,
For a proof outline, write pointwise and exchange the expected value and Lebesgue integral using Tonelli theorem; is an indicator function. The inclusive version has the same Lebesgue integral: positive probability masses occur at only a countable set of thresholds, which has Lebesgue measure zero. For a nonnegative integer-valued random variable , the tail-sum formula instead gives . The integer hypothesis is essential for these sums. The integral identity, its indicator function proof, and the integer version appear in Kyle Siegrist, Additional Properties of Expected Value: integration and summation of right tails.
For the fair die, this tail-sum formula yields , equal to the direct weighted arithmetic mean . For the rate-two exponential distribution, the area under the survival function is , its expected value. The first panel below shades an integral of the probability density function for a single threshold; the second shades an integral of the survival function across all nonnegative thresholds. These are analytical illustrations, with the displayed curves truncated at three; their exact total areas use an infinite integration range.
Figure 2.
Exponential model: one tail probability is density area, while expectation is total survival area
. Details.
Nonnegativity is essential to interpreting the upper tail probability area as : for a general real random variable that area equals the expected value of its positive part. An integrable random variable satisfies , subtracting the expected value of its negative part; infinite terms cannot be canceled. Even when is finite almost surely, its expected value can be infinite. For the shape-one Pareto distribution on , on and for , so the area is . The finite versus infinite expected value distinction is illustrated in Kyle Siegrist, Definitions and Basic Properties: exponential and Pareto expected values.
Why are tail probabilities useful? They answer threshold questions directly, and a concentration inequality can bound them without knowing the full probability distribution. For nonnegative with finite expected value, Markov inequality gives for : a large threshold relative to the expected value forces rarity. The quantile function reverses the question by choosing a threshold for a prescribed cumulative level. If with , then ; an atom of a measure may prevent equality. For the die, , with strict upper tail probability zero and inclusive upper tail probability .
A tail probability describes rarity under a specified probability distribution; it is not automatically a conditional probability after extra information. For with ,
because the threshold events are nested. Thus the die has but . Conditioning changes the sample space and renormalizes its probability measure; the same extreme outcomes can become common in that restricted population.
Tail sigma-algebra Created 2026-09-24 Updated 2026-10-14
For a sequence of random variables on a probability space , define
The decreasing sigma-algebras are generated by the listed random variables. Their intersection consists of events expressible after discarding every finite prefix. No independence is required.[ref]
The point is to distinguish a long observation record from a property of an infinite sequence. The event of seeing infinitely many heads and the event that sample means converge survive any finite change, while an exact first-head time does not. Kolmogorov's zero–one law adds the striking conclusion that under independence each tail event has probability zero or one; the tail sigma-algebra itself can still contain many nonempty sets.
Tail-sum formula Created 2026-09-24 Updated 2026-10-18
The tail-sum formula expresses an expected value as the area under a survival function. For every nonnegative random variable ,
This Lebesgue integral and the expected value may both be ; a probability density function is unnecessary. For a random variable taking values in the nonnegative integers, the corresponding sum is
The events and coincide because takes integer values. Thus the strict sum starts at zero, while the inclusive sum starts at one.[ref]
For an original example, let the probability mass function of be
Its expected value computed directly is . The strict tail probabilities at thresholds are : their sum is again . There is no probability mass at two, yet the event still has probability . The survival function has heights on the successive intervals ; each interval has length one, so its integral is the same sum.
Figure 1.
The same expected value from probability masses and survival columns
. The exact constructed probability mass function assigns to . The contribution appears in one survival column, while the contribution appears in three. Filled endpoints give the strict survival function at its jumps; open endpoints show the preceding height. The shaded area is . Details.
The indexing matters numerically. Starting the inclusive sum at zero adds , giving for this example. Starting the strict sum at one omits , giving . For a general nonnegative random variable, replacing with inside the integral is harmless: the difference is , nonzero at only a countable set of thresholds. For each positive integer , at most thresholds can have probability mass at least ; their countable union contains all thresholds with positive probability mass and has Lebesgue measure zero. This reasoning concerns an integral, not equality of the two survival function conventions at every point.
For a continuous original example, take the uniform distribution on the interval . Its survival function is for and zero for . The area is a triangle of base four and height one:
Directly integrating against the probability density function gives the same expected value, . The integral of the probability density function alone is one; it is the survival function whose area is the expected value.
Figure 2.
A uniform density has total probability one and survival area two
. Exact curves for the uniform distribution on . The probability density function has integral one, while the survival function has integral two. Splitting the survival area at gives before the cap and beyond it, the expected values of and the positive part , respectively. Details.
The proof counts layers of each outcome. For a realized value , the indicator function equals one over an interval of length , so
Tonelli theorem permits the interchange because the indicator function is nonnegative; no finite expected value is assumed. Similarly, an outcome contributes one indicator random variable for each of the events that occur:
Again Tonelli theorem applies to the nonnegative sum, or equivalently the monotone convergence theorem applies to its partial sums. This exchanges counting layers within each outcome with averaging across outcomes. The existing integer-valued hypothesis in the tail-sum formula explains what changes when the values are not integers; the tail integral formula for moments weights the layers to obtain higher moments.
For an integrable random variable that can have either sign, apply the nonnegative formula separately to its positive part and negative part :
Integrability makes both integrals finite. More generally the difference defines an extended expected value when at least one is finite; if both are infinite, subtraction does not define an expected value.[ref]
Tail probabilities are often easier to measure or bound than a whole probability density function. For a device's survival time, the survival function records the probability of still operating at each threshold; its integral gives the mean operating time. A finite observation window naturally gives the expected value of a capped nonnegative random variable. For a mathematical bound, for all implies . The tail-sum formula turns control of a whole family of events into control of an average.
Tail-sum formula for expectation Created 2026-09-24 Updated 2026-11-05
For a nonnegative random variable , Tonelli theorem gives
Consequently, for each , exactly when ; compare the integral on successive intervals of length .
Terminal wealth floor 2026-10-07
A terminal portfolio wealth floor is a prescribed lower bound, possibly a random variable, on terminal portfolio portfolio wealth. In a complete market, its state-price density cost cannot exceed initial portfolio wealth. Nonnegative admissibility replaces a possibly negative floor by its positive part.