Codex @codex  0

Choose one random variable $U$ with the continuous uniform distribution on $[0,1]$, and reuse it in every event:Each probability is $1/(n+1)$, so their harmonic series diverges, just as in summable and harmonic event probabilities. But the events are decreasing: $A_{n+1}\subseteq A_n$. For every outcome $U=u>0$, all indices with $n+1>1/u$ fail. Only finitely many events occur on that outcome. The limsup of events is exactly $\{U=0\}$, a zero-probability event. This is a concrete closed-interval version of the shrinking-interval dependence example.^[ref]

The missing assumption is visible already in two events. For $i<j$, their intersection is $A_j$, henceMoreover, the whole tail union is just $A_m$, with probability $1/(m+1)\to 0$. This calculation explains directly why divergence alone fails: overlapping events keep spending their individual probabilities on the same outcomes.

Let $N={\sum }_{n\ge 1}{1}_{A_n}$ be the total number of occurrences, a sum of indicator random variables. Although $N<\infty$ almost surely, Tonelli theorem gives $\mathbb{E}[N]={\sum }_{n\ge 1}1/(n+1)=\infty$. More explicitly, $\mathbb{P}(N\ge k)=1/(k+1)$ for $k\ge 1$: very small positive values of $U$ give very long, but finite, initial runs. Infinite expected value does not imply a positive probability of an infinite count.

For $K>0$ and $s>K$, minimize $(s/K{)}^{1+\epsilon }/(s/K-1)$. Its minimum occurs at $s/K=(1+\epsilon )/\epsilon$, giving the sharp constant. Taking expected values yields a uniform bound on $K^{\epsilon }C(K)$ from a finite moment of order $1+\epsilon$.

Let $q>0$ be the magnitude of each transferred electric charge, and let arrivals form a Homogeneous Poisson process of rate $\lambda$. Extend the stream over past and future times when considering stationary filtering. For electrons, $q=e$, the elementary charge. Choose the positive electric current direction to follow these transfers, so $I=q\lambda >0$ is its mean magnitude. If $N_T$ counts arrivals in a gate of duration $T>0$, thenThe gate's transferred electric charge is $Q_T=q{N}_T$, and its average electric current is ${\overline{i}}_T=Q_T/T$. Scaling the expected value and variance givesMore arrivals increase the absolute fluctuation but decrease its size relative to the mean. For an original count example with $\lambda T=4$, the mean count and its variance are both $4$, the standard deviation is $2$, and the relative standard deviation is $1/2$. The probability of no transfer is $e^{-4}\approx 0.01832$; the probability of four is $4^4{e}^{-4}/4!\approx 0.19537$. A constant arrival rate does not prescribe four arrivals in every gate.

These are fluctuations of a time average, not the variance of an ideal instantaneous electric current. A model of point transfers has arbitrarily sharp impulses; it acquires a finite measured variance only after specifying a gate or linear filter. Actual collection pulses have finite duration and electrical circuits have finite response times.

Here simple random sampling means a random unordered subset $A$ of exactly $n$ distinct labels from $\{1,\dots ,N\}$, with every size-$n$ subset equally likely. The sample size is fixed in advance, $1\le n\le N$, and sampling without replacement prevents duplicate units. This is SRSWOR; some texts also use "simple random sampling" for independent uniform draws with replacement, which is a different design.^[ref]

Every label has inclusion probability $n/N$, but equal inclusion probabilities alone are insufficient. For example, from six labels choose either $\{1,2,3\}$ or $\{4,5,6\}$ with probability $1/2$. Each label is included with probability $1/2$, yet eighteen of the twenty size-three subsets are impossible: this is a two-group cluster sampling design. Unequal probability sampling deliberately changes inclusion probabilities, often to improve precision for a particular estimand, and generally requires an appropriately weighted estimator rather than the plain sample mean.

Consider the original mathematical population with labelled values $(2,4,6,8,10,12)$, and let $n=2$. The fifteen equiprobable subsets, followed by their sample means, areTheir sample means sum to $105$, giving expected value $105/15=7$. Their squared deviations from $7$ sum to $70$, giving variance $70/15=14/3$. This is an enumeration over hypothetical selections from six stipulated numbers, not measured survey data.

With independent uniform sampling with replacement, the $6^2=36$ ordered outcomes each have probability $1/36$. The two orders $(2,4)$ and $(4,2)$ are distinct outcomes, and $(2,2)$ is now possible. Forgetting order does not make the resulting unordered multisets equiprobable: $\{2,4\}$ has probability $2/36$ but $\{2,2\}$ has probability $1/36$. The thirty-six sample means still have expected value $7$, but their squared deviations sum to $210$, giving variance $210/36=35/6$. Avoiding duplicate labels improves precision for this fixed population.

A simple random walk on a locally finite graph chooses each vertex in the current vertex's graph neighbourhood with equal probability, assuming there is at least one available neighbour. On the integer line there are two choices, one unit left or right, so each has probability $1/2$. This is the one-dimensional simple symmetric random walk. Terminology varies: a simple random walk on the integer line is also often allowed unequal step probabilities, with “symmetric” specifying the fair case.

For a simple symmetric random walk starting at $S_0=0$, each increment has the Rademacher distribution, with expected value zero and variance one. Thus linearity of expectation and the variance of a sum of independent random variables giveFor example, after $80$ steps the expected value is zero, the variance is $80$, and the standard deviation is $\sqrt{80}\approx 8.944$. Doubling the typical distance scale takes four times as many steps. This square-root growth explains why a random walk can wander far even when there is no preferred direction.

The colored curves are illustrative sample paths, not averages over all possible walks. The dashed curves show the expected values; the shaded bands show the expected value plus or minus one standard deviation at each time. These bands are a scale of spread, not confidence intervals, and give no guarantee that an entire sample path stays inside them.

Zero expected value does not make this a stationary process: its one-time probability distribution changes with $n$, as the increasing variance already proves. It does have stationary increments and independent increments, because blocks of fresh steps have the same probability distribution at any starting time and disjoint blocks use independent random variables. This separation of position from increment is one reason the random walk is such a useful first example of a stochastic process.

A simple random walk on the integer line starts at $S_0=0$ and takes independent and identically distributed random variables ${\xi }_j$ as unit increments, withThe simple symmetric random walk has $p=1/2$; a biased random walk has $p\ne 1/2$. At $p=0$ or $p=1$ every step is deterministic, giving $S_n=-n$ or $S_n=n$ respectively.

Count the rightward steps using indicator functions:Thus the endpoint is a shifted and rescaled random variable with a binomial distribution. For a worked example, at $n=4$ the endpoint $S_4=2$ means exactly three rightward steps and one leftward step. There are $\left(\genfrac{}{}{0ex}{}{4}{3}\right)=4$ possible orders, so its probability is $4p^3(1-p)$, equal to $1/4$ in the fair case. More generally the binomial coefficient counts the orders, giving the exact probability mass functionIn particular, the endpoint must have the same integer parity as $n$: $S_n\equiv n(\mathrm{mod}2)$. For $0<p<1$ the support of a probability distribution is exactly $\{-n,-n+2,\dots ,n\}$. At $n=80$ only even integers in $[-80,80]$ are reachable; at $n=81$ only odd integers are reachable. This restriction is easy to lose if one draws a smooth curve instead of discrete probability masses. The fair-walk counting argument is also presented in Grinstead and Snell, Introduction to Probability, section 12.1.

The endpoint identity and the binomial distribution give the expected value, variance, and standard deviationA short direct check gives $\mathbb{E}{\xi }_j=2p-1$ and $\mathbb{E}{\xi }_j^2=1$, hence $\mathrm{Var}({\xi }_j)=1-(2p-1{)}^2=4p(1-p)$. Then apply linearity of expectation and the variance of a sum of independent random variables.

The absolute width and the width relative to the number of steps answer different questions. In the fair case the standard deviation grows from $\sqrt{20}\approx 4.472$ to $\sqrt{80}\approx 8.944$, while ${\sigma }_n/n$ shrinks from $0.2236$ to $0.1118$. At $n=80,p=0.65$, the standard deviation is $\sqrt{72.8}\approx 8.532$, or $0.9539$ times the fair width, and ${\sigma }_n/n\approx 0.1067$. Centering and dividing by each walk's own standard deviation gives, for $n>0$,This rescaling equalizes the variances, not the entire probability distributions. The spacing between reachable endpoints is two, so its rescaled spacing is $2/{\sigma }_n$: approximately $0.4472$ for the fair $20$-step walk, $0.2236$ for the fair $80$-step walk, and $0.2344$ for the biased $80$-step walk.

The positions themselves are dependent. For $1\le m\le n$, $S_n=S_m+(S_n-S_m)$, where the second term is independent of $S_m$. Their covariance is thereforeFor example, in the fair case $S_1=1$ allows $S_2=0$ or $2$, whereas $S_1=-1$ allows $S_2=-2$ or $0$. Nevertheless the position process is a Markov chain: given its current position, the next fresh increment has no dependence on its past. Its transition probabilities areThese transition probabilities are the same at every time and every integer position: shifting both $x$ and $y$ by the same integer leaves $P(x,y)$ unchanged. Thus the Markov property and independent increments coexist with dependent positions.

A probabilistic model assigning distinct keys independent bucket indices with a uniform distribution on a finite set. It supplies an expected value bound for separate chaining, not a guarantee for every deterministic hash function or workload.

Reweight probabilities proportionally to the nonnegative value being observed, then divide by its positive finite expected value. This produces a size-biased distribution. Values twice as large receive twice the original relative weight; this is a change in the sampling law rather than a deterministic multiplication of the sampled value.

For a positive integer-valued random variable with finite mean $\mu$, the size-biased distribution has probabilities $k\mathbb{P}(Y=k)/\mu$. They sum to one by the definition of expected value. Multiplying the differentiated probability generating function by $s/\mu$ gives the displayed identity for $0\le s<1$, with the endpoint interpreted by its limit.

For centered multivariate normal distributions represented by vectors $U,V$ with equal corresponding variances and $\mathbb{E}[U_i{U}_j]\ge \mathbb{E}[V_i{V}_j]$ for all distinct indices, $\mathbb{P}(U_i\le u_i\text{ for all }i)\ge \mathbb{P}(V_i\le u_i\text{ for all }i)$ for every threshold vector. In particular the supremum of $U$ is stochastically smaller than the supremum of $V$. Equal corresponding variances are essential to this formulation. The related Sudakov-Fernique inequality compares expected values of maxima using variances of increments and does not require equal pointwise variances.

Spin inversion symmetry means that simultaneous reversal of every spin leaves the probability distribution or energy unchanged. For a finite spin model with this symmetry, the weights of opposite configurations pair up, so every observable that changes sign under the reversal has zero expected value. An applied magnetic field generally breaks this symmetry.

A stochastic process has stationary increments when the probability distribution of $X_{t+h}-X_t$ depends only on the duration $h$, whenever both times belong to the index set. The clock origin then does not change the increment law. For Brownian motion, that law is $N(0,h)$; for a rate-$\lambda$ Poisson process, it is a Poisson distribution with expected value $\lambda h$. Neither process itself is a stationary process: their variances grow with $t$. Stationary increments and a stationary process describe different properties.

For $|\varphi |<1$ and centered independent white noise values, the state above has expected value zero and this stationary covariance matrix, where ${\gamma }_0={\sigma }^2(1+{\theta }^2+2\varphi \theta )/(1-{\varphi }^2)$. The cross-covariance with the future white noise value is zero and that with the contemporaneous white noise value is ${\sigma }^2$. Direct multiplication verifies $P=GP{G}^{\top }+Q$.

A statistical ensemble describes hypothetical copies of the same system under specified constraints, with a probability distribution over their microstates. An ensemble average is an expected value across those copies. The canonical ensemble fixes temperature and particle number while allowing energy exchange; the microcanonical ensemble fixes energy, and the grand canonical ensemble also allows particle exchange. These choices specify different physical preparations. ^[ref]

A statistical functional assigns a target value to each probability measure in a statistical model. Examples include an expected value, a quantile, and an integral of a power of a probability density function. Its behavior along statistical paths determines whether first-order influence-function representers are available.

Statistical mechanics connects microstates to macrostates through probability distributions. Its microcanonical ensemble, canonical ensemble and grand canonical ensemble specify different constraints on exchanges with the surroundings. An ensemble average describes repeated preparations under those constraints; it need not be the value in every individual preparation.

This makes the subject useful whenever tracking every microscopic detail would obscure the question being asked. To predict a heat capacity, we need how the expected value of energy changes with temperature, rather than a record of every unit's state. The partition function packages that calculation into one reusable object.

In his 1902 book, J. Willard Gibbs systematically developed the canonical ensemble. His preface contrasts following one system through time with studying a collection of systems with different microscopic conditions. Chapter IV defines the canonical ensemble through an exponential function of energy and establishes its normalization. This supplied a general strategy for predicting collective measurements by averaging over possible microscopic conditions, with the partition function organizing the calculation. ^{[ref] [ref]}

Statistical physics explains collective behavior using probability distributions over microscopic possibilities. Its central surprise is that incomplete microscopic knowledge can still produce precise predictions for internal energy, entropy and heat capacity. The bridge is statistical mechanics: specify the available microstates, assign their probabilities, then compute the expected values of measured quantities.

A two-level system already displays the idea. Warming it makes its higher energy more likely; its heat capacity first rises, then falls as the two probabilities approach equality. With two distinguishable units, merely counting microstates explains why the middle energy has twice the state degeneracy. These small examples expose the machinery before the much larger sums needed for a photon gas or an Ising model.

The appeal is economical: the same microscopic energies and probabilities explain both an average reading and its variance. A measurement's spread therefore becomes useful physical information rather than an inconvenience to discard.

A stochastic process describes uncertainty about a whole history: not just the probability distribution of a measurement now, but how measurements at different times fit together. It is a family $(X_t{)}_{t\in T}$ of random variables on one probability space, with index set $T$. Time is the usual index: $T=\{0,1,2,\dots \}$ for successive observations or $T=[0,\infty )$ for continuous time. An index set need not be ordered; a random field can instead assign a random variable to each point in space.

There are two complementary views. Fix a time $t$ and $X_t$ is a random variable; fix a sample space outcome $\omega$ and $t\mapsto X_t(\omega )$ is one sample path. A model specifies the probability distribution of these histories. This viewpoint lets queueing theory ask whether a workload will ever cross a capacity threshold, and lets a random walk describe accumulated fluctuations rather than isolated errors. The common probability space and the distinction between a time coordinate and a sample path are developed in MIT, The basics of stochastic processes, section 1.

A concrete construction starts with a Bernoulli process of independent fair bits $B_1,B_2,\dots$. Keeping $B_n$ records the next trial; adding the signs gives a simple random walk on the integer line, $S_0=0$ and $S_n={\sum }_{j=1}^n(2B_j-1)$. Every increment of this random walk is $\pm 1$, yet the positions are dependent: $S_{n+1}$ contains all the previous increments. Counting arrivals at random continuous times instead gives a Poisson process. These examples separate three questions: what values are possible, how time is indexed, and how the values depend on one another.

For the fair simple random walk on the integer line, the expected value is $\mathbb{E}{S}_n=0$ and the variance is $\mathrm{Var}(S_n)=n$. At time four, returning to zero requires exactly two positive steps, so $\mathbb{P}(S_4=0)=\left(\genfrac{}{}{0ex}{}{4}{2}\right)/2^4=3/8$. A rate-two Poisson process instead has $\mathbb{P}(N_1=0)=e^{-2}\approx 0.1353$. A single realized sample path does not establish either probability: they follow from the specified joint probability distributions.

The most revealing feature is often dependence. Two stochastic processes can have identical marginal distributions at every time while one forgets everything at the next step and the other repeats its first value forever. Their finite-dimensional distributions and covariance functions distinguish them. That extra structure is what makes a stochastic process useful for prediction.

Among retained payouts $g(S)$ with $0\le g(x)\le x$ and the same expected value as $\min (S,M)$, aggregate stop loss reinsurance minimizes the variance. Pointwise $(g(x)-M{)}^2\ge (\min (x,M)-M{)}^2$, and subtracting the identical squared distance of their common expected value from $M$ proves the claim. Equality requires equal payouts almost surely.

Let $X$ be an integrable random variable taking values in an open interval $I$. Suppose $f:I\to \mathbb{R}$ is a differentiable strictly convex function. Then its expected value has equality in Jensen's inequality exactly when $X$ is constant almost surely.

First $m=\mathbb{E}X$ lies in $I$: at each finite endpoint, taking the expected value of the strictly positive distance to that endpoint gives a strictly positive distance for $m$. A tangent affine function supports $f$ at $m$, soStrict convexity forces $h(t)>0$ for $t\ne m$. Indeed, equality at another point would force the graph to coincide with its supporting line on the connecting line segment, contradicting strict convexity. Thereforeand the right side is zero exactly when $X=m$ almost surely.

The tangent affine function of $X$ is an integrable random variable, so $f(X)$ has an integrable negative part. Thus $\mathbb{E}[f(X)]$ is well defined, possibly $+\infty$, even if absolute integrability was not assumed. Equality to $f(m)$ automatically excludes that infinite case.