Random walk Created 2026-09-24 Updated 2026-10-14
A random walk is a stochastic process formed by accumulating random steps. In its additive form, the increments are independent and identically distributed random variables and the position after steps is
The steps can have more choices than and : they may be integers, real numbers, or vectors. A random walk on a graph instead describes steps between vertices along edges; its available moves can depend on the current vertex.
For example, starting at zero, the steps produce the positions . This is one sample path; other step sequences can have the same endpoint. A simple random walk on the integer line makes this distinction particularly tangible: repeated Bernoulli trials determine both a whole sample path and an endpoint with a binomial distribution. All positions below are dimensionless quantities measured in steps of length one, and counts discrete updates.
The model is useful because it separates an average tendency from accumulated fluctuations. A fair simple symmetric random walk has zero expected value at every time, yet its standard deviation grows: cancellation across many possible sample paths does not keep each walker near zero. The two figures below compare sample paths with exact endpoint probabilities. For the basic additive construction and its independent increments, see MIT, Stochastic Processes I, section 2.
For players with independent valuations having a continuous distribution function , a type occupies rank when exactly rivals have higher values. Multiplying this binomial distribution by the rank prize and summing gives . Ties occur only on zero-probability events when the valuation law is atomless.
A box contains one certainly red ball and independently coloured extra balls, each red with probability . Conditional on the first draw being red, a second draw without replacement is red with probability . To see the effect of using a conditional probability, let , where has a binomial distribution with parameters . The conditional probability is : histories with more red balls are more likely to produce the first red draw and therefore receive greater weight.
Repetition code Created 2026-10-03 Updated 2026-10-14
A binary repetition code represents one bit by repeating it times, using codewords and . Their Hamming distance is , and the code rate is . For odd , the majority function recovers the input whenever fewer than half the positions have flipped. Even allows a tied vote, so a decoder must specify how it handles that case.
For , the received words decode to zero; decode to one. The decoder therefore corrects every single flip. This guarantee concerns the number of flipped positions, independent of any probabilistic model.
If each position passes independently through a binary symmetric channel with crossover probability , a decoded error requires at least two flips. The binomial distribution gives
At , the uncoded error probability is , while the three-bit code's decoded error probability is . This improvement costs three transmitted bits per message bit. For , ; for , one should first exploit the known tendency to invert symbols rather than applying the same majority rule blindly.
Figure 1.
Three-bit majority decoding corrects one flip and lowers the error probability at the cost of rate
. Details.
Independent flips are an essential assumption for the displayed formula. If a single common disturbance flips all three positions together with probability , repetition provides no improvement. More efficient error-correcting codes distribute redundancy across larger message blocks; repetition is useful as a transparent example of the geometry and the reliability–rate tradeoff.
Return period Created 2026-10-06 Updated 2026-10-14
A return period expresses how rare a threshold exceedance is in a specified time block. Let be the sample maximum in block , fix a threshold , and suppose is the same for every block. The conventional return period is blocks; blocks of duration give a time of . Thus an annual exceedance probability means a 100-year return period. This names a frequency scale, not a schedule or a guarantee of one exceedance in each century. The annual convention and the misleading “once every 100 years” interpretation are discussed in USGS: the 100-year flood.
To derive a waiting-time interpretation, assume the exceedance indicators are independent and identically distributed random variables with the Bernoulli distribution of parameter . Define , counting the successful block as well as all preceding blocks. The first Bernoulli trials must fail and the th must succeed, so independence gives the geometric distribution
The expected value can also be read directly from the geometric series: . This counts blocks from the starting boundary, not the exact time of an exceedance within its block. Counting only failed blocks instead would give expected value . The waiting law and its memorylessness of the geometric distribution are developed in Kyle Siegrist, Random Services: the geometric distribution.
For a horizon of complete years, at least one exceedance is the complement of no exceedances. Under the same independence and constant- assumptions,
For the toy choice , this gives in 10 years, in 30 years, in 50 years, and in 100 years. Even after 100 years there is a chance of no exceedance. These numbers are exact model calculations rounded to one decimal place, not measured frequencies. The median waiting count is 69 years, since ; the expected value is 100 years. A mean and a median answer different questions.
Figure 1.
A 1 percent annual exceedance probability accumulates across independent years
. Exact toy probabilities for . The rising curve uses independent annual Bernoulli trials; the flat line uses one shared Bernoulli random variable for every year. Both have the same annual probability, but different multi-year probabilities. No observations are plotted. Details.
The exceedance count has the binomial distribution under independence, with expected value . Thus has expected value one, although its probability of being at least one is only : some realizations contribute several exceedances. For , the memorylessness of the geometric distribution gives . A long quiet spell does not make the next independent year's exceedance overdue, and an exceedance this year does not force a quiet interval. Two specified consecutive years both exceed with probability in this toy model.
A reproducible tabletop experiment makes the assumptions tangible. Put 100 identical slips numbered 1 to 100 in a bag; after thoroughly mixing, draw one and record whether it is 100. Replace the slip and mix again before every draw. Treat each draw as one toy year, so the exceedance indicator is a Bernoulli random variable with , provided the sampling is uniform and successive draws are independent. Repeat complete 30-draw runs and record the fraction containing at least one 100; separately repeat runs stopped at the first 100 and record their waiting counts. The first fraction targets ; the waiting-count histogram targets . Replacement keeps fixed, mixing is intended to support independence, and repeated runs expose sampling variation. This is a proposed demonstration of the toy model; no experimental results are asserted here.
Stationarity supports a time-invariant annual probability distribution, but does not imply independence. For an extreme counterexample, draw just one Bernoulli random variable of parameter and set for every year. This is a stationary process, yet the probability of any exceedance in every finite horizon is ; from an arbitrary starting boundary the wait is infinite with probability . The reciprocal marginal frequency therefore need not equal that forward waiting expected value. With equal marginal probabilities and arbitrary dependence, only the general bounds follow, the upper bound being the union bound. Linearity of expectation still gives without independence. Mere zero correlation does not restore the product formula; even pairwise independence does not imply joint independence. If annual probabilities change to , independence instead gives , and one fixed return period no longer describes all years.
For independent and identically distributed Bernoulli random variables with success probability , the sum has a binomial distribution. Every length- pattern with successes has probability , and the binomial coefficient counts its possible success positions. Therefore
The usual endpoint convention includes and as deterministic cases. For and , the five probability masses are divided by . The sample mean takes values , with variance and standard deviation . At , its variance is and its standard deviation is .
Figure 1.
Exact sample-mean distributions distinguish fresh IID draws from a copied common coin
. Original exact binomial distribution probabilities for sample sizes . Blue marks give the mass at each possible mean under independent fair draws. Orange marks give the masses for the same fair draw copied times: at zero and at one for every . These are discrete probability masses, not density values or measured histograms. Details.
Draw two different labels uniformly from using sampling without replacement. Each position has a uniform distribution on a finite set, but . Two independent random variables with those same marginal distributions would match with probability . Equal marginal laws therefore do not make this sample an IID sample. Mark some of those four labels as successes. Their count in the two draws has a hypergeometric distribution; independent Bernoulli trials with a common success probability give a binomial distribution.
Scaled deviance 2026-10-14
For an exponential dispersion family with a common dispersion parameter , scaled deviance compares a fitted model with the saturated statistical model at the same dispersion:
For a normal distribution with variance , unscaled deviance is the residual sum of squares and scaled deviance is that sum divided by . For the binomial distribution, and the two coincide. Unknown Gaussian dispersion requires variance estimation; raw deviance differences should not be treated as chi-squared statistics without the scale.
Schmidt projection entanglement concentration Created 2026-10-14 Updated 2026-10-20
For copies of a normalized pure state , regroup Alice's and Bob's tensor factors. Alice performs a projective measurement of the total number of zeroes, preserving all coherence among strings with that count. The outcome has binomial distribution
Each retained string has the same complex amplitude , so, up to a common phase, the conditional quantum state is
Its equal Schmidt coefficients make it a maximally entangled state of Schmidt rank . Alice sends the count to Bob. They can relabel the corresponding local computational basis strings with local unitary operations, and convert the resulting resource to Bell pairs. Measuring the individual bits instead would reveal the entire string and leave a product state.
Second moment method Created 2026-09-24 Updated 2026-10-14
The second moment method uses a random variable's expected value and second moment to establish that a counted or measured quantity is positive with substantial probability. It is useful when a positive average alone could be caused by extremely large values on very rare outcomes. Controlling the variance rules out too much of that concentration in rare outcomes.
For a nonnegative random variable with and finite second moment, the second-moment bound for positivity gives
Equivalently, . The method also includes the simpler Chebyshev's inequality bound . Thus a sequence of such random variables with has with probability tending to one. These statements require no independence assumption. The variance argument and its use for random counts are developed in Zhao's lecture notes. [ref].
For an original toy count, let count successes in ten independent Bernoulli trials with success probability . The binomial distribution gives
Consequently . The exact probability is . The first moment method and union bound only give the upper bound here. First and second moments therefore answer complementary existence questions.
Figure 1.
The second moment method certifies a positive count, while statistical dependence can change the conclusion despite an unchanged mean
. Left: exact binomial distribution probabilities of at least one success and second-moment lower bounds for ten independent trials as the success probability varies. Right: the same ten trials with either occur independently or share one common Bernoulli outcome. Both counts have mean one, but the perfectly linked count has much larger variance and only probability of being positive. All data are original specified probability models. Details.
For a sum of indicator random variables, the variance of a sum is
Independence makes the covariances vanish, but the formula itself also covers dependent counts. For the same ten success probabilities, let every trial share a single Bernoulli random variable : then , with mean one, variance nine, and . The second-moment lower bound is exactly . Ignoring the positive covariances would produce the false independent-count conclusion. Zhao's discussion of sums of indicators explains why overlapping counted structures require these pairwise terms. [ref].
The Fano factor compares count variance with its mean over a specified gate. For a stationary unidirectional stream with fixed charge , finite count second moments, and a generalized autocovariance with finite integral so that the long-gate limit exists,
The zero-frequency relation concerns the continuous fluctuation spectrum, excluding the mean spectral line. It follows by writing as the double integral of the current autocovariance; for long , its leading term is . A finite-gate , a frequency-dependent ratio , and a factor inferred from one filtered variance are not interchangeable. The last is weighted by the instrument's .
Two original counting counterexamples show why electric charge discreteness is insufficient. In independent time slots, transfer one charge with probability in each slot: the binomial distribution gives mean , variance and Fano factor . A random common origin for the slots removes a preferred time origin; the long-gate factor remains . Alternatively, if independent Poisson process bursts each transfer exactly two charges, then gives and factor . Both count individual charges; only the first model allows at most one per slot, and the second groups them in pairs.
Consequently is a reference for independent unidirectional arrivals, not a universal noise law for all electrical conductors. Correlations, charge multiplication, and bidirectional transfer require a more specific model. For example, independent opposite streams with positive current magnitudes and have net mean but one-sided fluctuation spectrum . Using would miss their fluctuations when the net current is small. At thermal equilibrium, the full transport model must also respect thermal electrical noise; adding unrelated phenomenological formulas can double count the same underlying transfers.
Simple random walk on the integer line Created 2026-09-24 Updated 2026-10-14
A simple random walk on the integer line starts at and takes independent and identically distributed random variables as unit increments, with
The simple symmetric random walk has ; a biased random walk has . At or every step is deterministic, giving or respectively.
Count the rightward steps using indicator functions:
Thus the endpoint is a shifted and rescaled random variable with a binomial distribution. For a worked example, at the endpoint means exactly three rightward steps and one leftward step. There are possible orders, so its probability is , equal to in the fair case. More generally the binomial coefficient counts the orders, giving the exact probability mass function
In particular, the endpoint must have the same integer parity as : . For the support of a probability distribution is exactly . At only even integers in are reachable; at only odd integers are reachable. This restriction is easy to lose if one draws a smooth curve instead of discrete probability masses. The fair-walk counting argument is also presented in Grinstead and Snell, Introduction to Probability, section 12.1.
Figure 1.
Exact binomial endpoint probabilities for fair walks of 20 and 80 steps and a biased walk of 80 steps; only even endpoints are possible.
Details.
The endpoint identity and the binomial distribution give the expected value, variance, and standard deviation
A short direct check gives and , hence . Then apply linearity of expectation and the variance of a sum of independent random variables.
The absolute width and the width relative to the number of steps answer different questions. In the fair case the standard deviation grows from to , while shrinks from to . At , the standard deviation is , or times the fair width, and . Centering and dividing by each walk's own standard deviation gives, for ,
This rescaling equalizes the variances, not the entire probability distributions. The spacing between reachable endpoints is two, so its rescaled spacing is : approximately for the fair -step walk, for the fair -step walk, and for the biased -step walk.
The positions themselves are dependent. For , , where the second term is independent of . Their covariance is therefore
For example, in the fair case allows or , whereas allows or . Nevertheless the position process is a Markov chain: given its current position, the next fresh increment has no dependence on its past. Its transition probabilities are
These transition probabilities are the same at every time and every integer position: shifting both and by the same integer leaves unchanged. Thus the Markov property and independent increments coexist with dependent positions.
A sufficient statistic allows the observed likelihood function to be stored through a shorter record, up to a parameter-independent factor. For the Bernoulli sample sum as a sufficient statistic, write
The second likelihood function belongs to the observed count; the first belongs to the observed sequence. Their absolute values differ by the binomial coefficient, but their likelihood ratios between any two interior parameter values agree. A maximum-likelihood estimator can therefore use just .
Figure 1.
Sequence to count: sufficient compression preserves likelihood comparisons
. For , three example sequences share . Recording produces the count likelihood . Normalizing each curve by its maximum makes the two curves coincide. A reverse random draw from the six arrangements restores the full-data probability distribution, using no unknown . Details.
This compression discards which arrangement actually occurred. To reproduce the original experiment's probability distribution, first draw from its binomial distribution and then draw uniformly from the arrangements with that sum. The second step is independent of , so it supplies no additional evidence about . It does not recover the particular discarded sequence. If the scientific question changes to time trends or statistical dependence between observations, the common- independent and identically distributed Bernoulli model may no longer apply, and the ordering can become informative.
Stirling formula Created 2026-09-24 Updated 2026-10-14
Stirling's formula turns the long product defining a factorial into a short expression. For positive integers , put
Then as . This is asymptotic equivalence: , so the relative error tends to zero. It does not say that the absolute error tends to zero, and the formula is not an equality for finite . The Gamma function extends this approximation beyond integer inputs. [ref]
For ,
The leading estimate is about low. Multiplying by the first correction factor gives
which is about low. The Stirling correction by Gaussian moments explains why this factor is rather than an arbitrary numerical adjustment.
Figure 1.
Relative factorial errors shrink faster after the first Stirling correction
. For each integer , the curves compare and with the exact factorial. The vertical axis is the nonnegative relative error, on a logarithmic scale. Details.
The usefulness is in ratios as well as in enormous numbers. Applying Stirling's formula to the three factorials in a central binomial coefficient gives
This counts the ways to choose exactly half of objects. Equivalently, for independent fair coin tosses, the binomial distribution assigns probability to exactly heads. An exponentially large count becomes a simple probability scale after cancellation.
For numerical calculations, taking the natural logarithm yields
where . Computing a logarithm of a large factorial avoids representing the factorial itself. The Robbins bounds for Stirling's formula give a finite- interval for this logarithmic remainder, rather than only a limiting statement.
The factor is beautiful because a Gaussian integral lies behind a discrete product. The Gaussian peak behind Stirling's formula displays this connection through the Gamma integral and Laplace's method. A rigorous proof must also control the tails of the integral; the local expansion explains the geometry but is not by itself a complete proof.
James Stirling published Methodus differentialis in 1730, a treatise on summation and interpolation of infinite series. This setting connects the formula with a practical calculation problem: taking logarithms turns a long product into a sum, whose growth can be approximated. [ref]
Strong law of large numbers Created 2026-09-24 Updated 2026-10-14
Let be real independent and identically distributed random variables with , and let be their expected value. Their sample mean satisfies
This is almost sure convergence: with probability one, an entire infinite sequence of observations has averages that eventually stay within every prescribed positive tolerance of . The variance need not be finite. The strong law of large numbers implies the weak law of large numbers, which asserts convergence in probability at each increasing sample size.
For independent trials with a Bernoulli distribution of parameter , is one for success and zero for failure. Then is the observed success proportion, and the strong law of large numbers says almost surely. This turns a theoretical probability into a quantity one can estimate by repetition. It does not say that each new trial improves the estimate: a long string of failures can move the proportion away from before later observations bring it back.
Figure 1.
Bernoulli averages approach their mean: sample paths and exact error probabilities
. The left panel shows four reproducibly simulated sequences of independent Bernoulli trials with success probability . The right panel calculates from the exact binomial distribution and compares it with the bound from the Chebyshev inequality. Small irregularities reflect integer-valued counts. The paths illustrate almost sure convergence; a finite simulation cannot prove it. Details.
The strong law of large numbers gives the limiting location of the sample mean, while the central limit theorem describes its fluctuations when the variance is finite and positive. The two assertions can hold together:
The first error shrinks to zero; the second has been enlarged enough to retain a nontrivial probability distribution. The second assertion is convergence in distribution, not a claim that the enlarged error along one sequence approaches a particular normal random variable.
The assumptions make a practical difference. If every observation repeats the same nonconstant Bernoulli random variable, , then for all : identical probability distributions alone do not make repeated observations informative. For independent observations with a Standard Cauchy distribution, the Cauchy sample mean has an unchanged distribution, so it does not concentrate around a constant. Its expected value is undefined, and ; neither example contradicts the stated strong law of large numbers.
A brief proof outline in the case of finite variance uses the Kolmogorov maximal inequality for the centered partial sums. The probability that their maximum up to exceeds is at most . These bounds are summable, so the Borel-Cantelli lemma controls all sufficiently large such blocks and hence every intermediate sample size. The general case requires truncation to control large observations; the argument using finite variance alone does not prove that full version.
The same stopping rule connects a waiting count to a fixed-length count. Let be the trial number of the th success, and let count successes in the first trials, so has a binomial distribution. For any nonnegative integer ,
The target is still unmet after trials exactly when fewer than successes have occurred. With , and , this gives ; equivalently, , matching . At , the event identity still holds, with and deterministically.
Success-failure stopping identity Created 2026-10-18 Updated 2026-10-20
In independent Bernoulli trials with success probability and failure probability , let be the trial of success and the trial of failure , for positive integers . If success occurs after failures, its final trial is a success and the preceding trials have successes and failures. The negative binomial stopping argument therefore gives the title sum.
Put . By trial , at least one target must have been reached, so the same event is that a binomial distribution with trials has at least successes. Conditioning instead on the final trial of a block of trials gives
Indeed, more than successes in the full block guarantee the success target won; with exactly successes it won only when the final trial was a failure. These disjoint cases explain the extra boundary term. A zero success target wins immediately; a zero failure target has already been reached. To include in the displayed finite sum requires the explicit generalized convention ; an upper summation limit of denotes an empty sum.
Synaptic vesicle 2026-10-14
A synaptic vesicle is a small membrane-bounded compartment in a presynaptic terminal that stores neurotransmitter for release. Fusion with the cell membrane exposes its contents to the synaptic cleft; the packet supplied by one synaptic vesicle is a quantum of release. Quantal neurotransmitter release refers to these discrete packets, rather than the release of one molecule at a time. The postsynaptic response to a packet can vary with its contents, receptors and recording conditions. See Purves and colleagues: vesicles as the source of transmitter quanta.
The useful cycle is filling, preparation for fusion, calcium-triggered exocytosis, and recovery of membrane and proteins through endocytosis for reuse. Different recovery routes occur; recycling need not preserve a particular synaptic vesicle as an identifiable container throughout. Limited availability of prepared synaptic vesicles can reduce release during repeated stimulation, while changes in synaptic release probability can also change successive responses. See Basic Neurochemistry: secretion machinery and vesicle reuse and 3.1 Synapses.
For an illustrative calculation, assume four release sites, each with one prepared synaptic vesicle, each releasing with synaptic release probability during one presynaptic action potential. Assume independent Bernoulli trials and at most one release per site. The released packet count has a binomial distribution:
The probability of no release is ; at least one release has probability . An expected value of one packet does not mean that every action potential releases one packet. If each packet contributes a stipulated conductance increment and increments add, the expected value of that increment is , with zero increment on about of trials. These assumptions are a teaching model: shared calcium fluctuations, unequal sites, depletion and nonlinear receptor responses can invalidate them.
Let be independent random variables, each with the Bernoulli distribution of parameter , and let . Multiplying their characteristic functions and expanding by the binomial theorem gives
The coefficients are exactly the probability mass function of the binomial distribution with parameters : the sum takes with probabilities . Direct outcome counting verifies them: an outcome with ones has probability , and there are such outcomes. The coefficients sum to one, recovering .
At , . This is also : the complex exponential now records parity. The moments agree by two routes: linearity of expectation gives , while the variance of a sum of independent random variables gives . Summing gives the second moment , also recovered as .
Figure 1.
Independent products and the different limits of averages and variance-normalized sign sums
. Top: exact real parts and imaginary parts of and its cube, the characteristic function of the three-variable binomial distribution. Bottom: for independent symmetric signs, approaches one under averaging, whereas approaches under variance normalization. All curves are exact model formulas; the finite curves illustrate the two limits rather than proving a general central limit theorem. Details.
For a two-component mixture model with known component tail probabilities above a threshold, the overall probability is . An empirical exceedance fraction gives the displayed method-of-moments estimate when . Under independent observations, the exceedance count has a binomial distribution, and its constrained maximum-likelihood estimate is the same expression clipped to . Equal component tail probabilities make this summary uninformative about .