Put for . Independence of the underlying exponentials gives andBy the covariance of overlapping independent products, adjacent terms have covariance one. Terms separated by at least two indices use disjoint independent factors, so their covariance is zero. There are adjacent pairs. Linearity of expectation and the variance of a sum formula therefore yieldIn particular, the products themselves are not independent merely because the original are.
Let be the indicator random variable that colour appears at least once. Then . For a colour of single-roll probability , independence of rolls gives . By linearity of expectation,The three indicators are not independent, but linearity of expectation does not require them to be. This is the expected number of observed categories.
Let be the indicator that black bead belongs to the white bead's connected component. The number of such black beads is . By linearity of expectation, no independence between these indicators is needed.
By linearity of expectation, . Expanding the square and again using linearity of expectation givesHere the zero means imply and . The covariance expansion itself does not require independence. Under the additional independence stated here, , so it simplifies further to .
The finite-partition form of the law of total expectation statesBecause the events form a partition of a sample space of the sample space, pointwise. Multiply by and use linearity of expectation to obtainAn event of zero probability contributes zero to the middle expression and is omitted from the final expression, so no undefined conditional expectation is multiplied by zero.
Write and . Since , the latter expected value is finite. Conditional on the first move to or , total time is one plus the remaining time, while the terminal success indicator random variable is unchanged. The Markov property and linearity of expectation giveAt either absorbing boundary the remaining time is zero, so .
Let . The linear term has zero second difference and has second difference . Consequentlywhich is exactly the required recurrence, and . Uniqueness follows as before: the difference of two solutions has zero second difference and zero endpoint values. HenceThis is the success-weighted duration underlying the duration of gambler's ruin conditioned on a chosen boundary.
Assume finite second moments and set . By linearity of expectation, centring gives . Expanding its square and using the definitions of variance and covariance,No independence assumption is required for this variance of a sum identity.
By linearity of expectation, . The variance of a sum and the negative covariance just proved giveApply Chebyshev's inequality to the empty-box indicators in uniform allocation:This proves the required concentration of the empty-box fraction. Since , it also yields convergence in probability of to .
First , so linearity of expectation gives . The tail integral formula for expectation, followed by the finite binomial theorem expansion, givesThe exchange involves only a finite sum. As a useful check, substitution rewrites the integral as , the harmonic number .
Apply the law of total expectation to the bounded random variable . Linearity of expectation givesThis also holds for the first step from .
Poisson binomial distribution 2026-10-14
The sum of independent Bernoulli random variables with possibly unequal success probabilities has a Poisson binomial distribution. Its probability generating function is : expand the product to read off the probability mass function. Linearity of expectation and additivity of variance for independent random variables give and . Equal recover the binomial distribution.
For three trials with success probabilities , expanding gives masses at . Their expected value is and variance is . Replacing all three success probabilities by their average preserves the expected value but gives binomial distribution variance , so it does not preserve the law.
Poisson limit for occupancy fractions Created 2026-10-06 Updated 2026-10-25
In a uniform balls-in-bins allocation with and , a fixed bin count has convergence in distribution to a Poisson distribution variable of mean by the Poisson limit theorem. If counts bins containing exactly balls, symmetry and the linearity of expectation give for each fixed . In particular the expected empty fraction tends to , and the expected fraction with at least two balls tends to . These are statements about expectations; convergence of the random fractions requires an additional argument.
Random cut lower bound Created 2026-10-03 Updated 2026-10-14
Every finite graph with edges has a cut of a graph of size at least , and hence a spanning bipartite graph with at least edges. Put each vertex independently on either side with equal probability. Every edge crosses with probability , so linearity of expectation gives expected cut size .
Random variable Created 2026-09-24 Updated 2026-10-14
A real-valued random variable is a measurable function from a probability space to the real numbers with their Borel sigma-algebra. More general targets give random elements. For every Borel set , measurability ensures that is an event, so the question "does the observed value lie in ?" has a defined probability.
The function is fixed; its input outcome is uncertain. This separates what we measure from how the uncertainty is modeled. A probability distribution is the resulting pushforward measure, . A cumulative distribution function, probability mass function, or probability density function describes that measure in an appropriate setting. The random variable and its probability distribution are different objects: the former also retains its relationship to other measurements on the same sample space.
For two independent uniform dice, let record their results and define and . The 36 ordered pairs are equally likely, but the resulting probability mass functions differ:For the sum, count the diagonal of the outcome grid having total . For the maximum, subtract the pairs with both results below from the pairs with both results at most . These examples show how a deterministic function transforms a simple input probability distribution into a nonuniform output one.
The corresponding expected values are and . The first follows immediately from linearity of expectation; the second follows from . Neither number must be an attainable result. An expected value describes an average under the probability measure, not a prediction that the next observation will equal it.
The distinction between a random variable and its probability distribution matters for dependence. and have the same discrete uniform distribution and are independent random variables. and a second copy of also have that same marginal probability distribution, but they always agree. Their joint probability distribution is concentrated on the diagonal; the marginals alone cannot say which relationship holds. This is why a model for several measurements must specify their joint law or an appropriate independence assumption.
Indicator random variables turn events into numbers: equals one on and zero elsewhere, and . This connects event counting with expected value and variance, and lets tools for random variables answer questions about probabilities. More complicated functions of observations can describe totals, waiting times, or estimation errors using the same framework.
Return period Created 2026-10-06 Updated 2026-10-14
A return period expresses how rare a threshold exceedance is in a specified time block. Let be the sample maximum in block , fix a threshold , and suppose is the same for every block. The conventional return period is blocks; blocks of duration give a time of . Thus an annual exceedance probability means a 100-year return period. This names a frequency scale, not a schedule or a guarantee of one exceedance in each century. The annual convention and the misleading “once every 100 years” interpretation are discussed in USGS: the 100-year flood.
To derive a waiting-time interpretation, assume the exceedance indicators are independent and identically distributed random variables with the Bernoulli distribution of parameter . Define , counting the successful block as well as all preceding blocks. The first Bernoulli trials must fail and the th must succeed, so independence gives the geometric distributionThe expected value can also be read directly from the geometric series: . This counts blocks from the starting boundary, not the exact time of an exceedance within its block. Counting only failed blocks instead would give expected value . The waiting law and its memorylessness of the geometric distribution are developed in Kyle Siegrist, Random Services: the geometric distribution.
For a horizon of complete years, at least one exceedance is the complement of no exceedances. Under the same independence and constant- assumptions,For the toy choice , this gives in 10 years, in 30 years, in 50 years, and in 100 years. Even after 100 years there is a chance of no exceedance. These numbers are exact model calculations rounded to one decimal place, not measured frequencies. The median waiting count is 69 years, since ; the expected value is 100 years. A mean and a median answer different questions.
The exceedance count has the binomial distribution under independence, with expected value . Thus has expected value one, although its probability of being at least one is only : some realizations contribute several exceedances. For , the memorylessness of the geometric distribution gives . A long quiet spell does not make the next independent year's exceedance overdue, and an exceedance this year does not force a quiet interval. Two specified consecutive years both exceed with probability in this toy model.
A reproducible tabletop experiment makes the assumptions tangible. Put 100 identical slips numbered 1 to 100 in a bag; after thoroughly mixing, draw one and record whether it is 100. Replace the slip and mix again before every draw. Treat each draw as one toy year, so the exceedance indicator is a Bernoulli random variable with , provided the sampling is uniform and successive draws are independent. Repeat complete 30-draw runs and record the fraction containing at least one 100; separately repeat runs stopped at the first 100 and record their waiting counts. The first fraction targets ; the waiting-count histogram targets . Replacement keeps fixed, mixing is intended to support independence, and repeated runs expose sampling variation. This is a proposed demonstration of the toy model; no experimental results are asserted here.
Stationarity supports a time-invariant annual probability distribution, but does not imply independence. For an extreme counterexample, draw just one Bernoulli random variable of parameter and set for every year. This is a stationary process, yet the probability of any exceedance in every finite horizon is ; from an arbitrary starting boundary the wait is infinite with probability . The reciprocal marginal frequency therefore need not equal that forward waiting expected value. With equal marginal probabilities and arbitrary dependence, only the general bounds follow, the upper bound being the union bound. Linearity of expectation still gives without independence. Mere zero correlation does not restore the product formula; even pairwise independence does not imply joint independence. If annual probabilities change to , independence instead gives , and one fixed return period no longer describes all years.
Sample mean under simple random sampling 2026-10-14
Fix real population values . Define the finite population mean and two normalizations of the finite population variance byThe values are fixed; only the selected subset is random. For its sample mean , the exact design expectation and variance areThe sample mean is an unbiased estimator of the finite population mean. At , every unit is observed and exactly, so the variance is zero. The case is deterministic and is handled directly, since is undefined then.[ref]
Chapter 2: Simple Random Sampling. Sampling Theory, Chapter 2, PDF pp. 2, 6–9: variance normalizations, unbiasedness, covariance derivation and finite population correction
The finite population correction depends on the baseline. With the normalization it is multiplying . With the normalization it is multiplying , the exact variance for independent uniform sampling with replacement. The standard error uses the square root of the relevant variance factor. Mixing the two normalizations gives a wrong answer even at , when replacement cannot matter. For the six-value example, , , and , reproducing without replacement and with replacement.
A short proof exposes the statistical dependence. Let be the value at position in uniform sequential sampling without replacement, so each has expected value and variance . Write . Every ordered pair of distinct labels is equally likely, and , hence for ,Linearity of expectation proves unbiasedness without requiring independence. The variance of a sum includes all ordered covariance terms, givingFor the original population, each pair of distinct draw positions has covariance . Drawing a high value leaves a lower mean among the remaining values, balancing subsequent draws. Equal marginal probability distributions therefore coexist with negative covariance.
When , the sample variance is an unbiased estimator of . To check this independently, use : its expected value is . Thus unbiasedly estimates the variance of the sample mean; its square root estimates the standard error, but need not be unbiased.[ref]
Chapter 2: Simple Random Sampling. Sampling Theory, Chapter 2, PDF pp. 10–12: sample-variance and standard-error estimation
Exact mean variance and expected distinct-unit coverage as sample size grows in the six-value population
. Details. The diagram compares variance with the expected fraction of distinct population labels observed. Without replacement this fraction is exactly ; with independent uniform sampling with replacement, a label is missed with probability , so linearity of expectation gives expected fraction . At six draws, replacement covers only about of labels on average and retains sample mean variance ; sampling all six distinct labels covers the population completely and removes selection uncertainty. This population coverage is distinct from confidence interval coverage probability.
The correction matters when the sampling fraction is appreciable. It calibrates uncertainty about this fixed population under the specified selection mechanism. It does not correct unequal selection weights, missing observations or measurement errors; observing every label also does not eliminate uncertainty about a larger unobserved population. Moreover, an exact variance alone does not guarantee the coverage probability of a proposed confidence interval: a small discrete sampling distribution can make a normal approximation inaccurate.
Semicoloring by independent hyperplanes 2026-10-05
Suppose a graph with vertices and edges has a vector coloring with adjacent inner products at most . Take independent copies of random hyperplane rounding and use the signs as a color. Each edge remains monochromatic with probability at most , so the expected number of monochromatic edges is at most by linearity of expectation.
Choose . Then Markov inequality gives . On success, delete one endpoint of every monochromatic edge. This random alteration method leaves at least vertices properly colored withThe removed vertices can retain their original colors, as the semicoloring only requires the retained induced subgraph to be proper. A failed draw can be detected and repeated, with at most two trials on average. The case needs just one color. This is the elementary independent-hyperplane construction in Karger, Motwani and Sudan's paper on approximate graph coloring.
Separate chaining Created 2026-10-13 Updated 2026-10-20
With separate chaining, each hash bucket in a hash table contains a collection of records, often a linked list. Lookup computes the bucket index, then follows that chain until a key equality comparison succeeds or the chain ends. A hash collision therefore affects the search work without merging unequal keys. Deletion searches the same chain and unlinks the matching record; in a singly linked list, remembering the preceding record during the search permits the unlink.[ref]
For the four-key integer example in hash table, insert a new record at the head of its chain after checking that its key is absent. Inserting in that order makes hash bucket hold , hash bucket hold , and the other two hash buckets empty. All five subsequent operations below select hash bucket because :
Separate chaining traces lookup, replacement, deletion and insertion through key equality
. Original deterministic model with . The table counts actual keys inspected, not elapsed machine time. Operations occur from top to bottom; only an absent key adds a new record, and the untouched record remains in hash bucket . Details.get(5) rejects key , accepts key , and returns after two key equality comparisons. get(13) rejects and reports absence after three comparisons. put(5,B*) examines and replaces the value without changing . delete(9) examines just the head record, removes it, and gives . Finally, put(13,E) rejects before inserting the new head record; returns to and the chain is . The successive hash table load factors are . These counts follow from the stipulated chain order; a different insertion policy changes the successful-search counts.For a chain of length , lookup and deletion by key take time complexity in Big O notation. The same applies to
put before any resizing, since replacement semantics require checking for an existing equal key. Head insertion alone is if absence is already known; that is a different precondition. These bounds assume bucket-index hashing, array access, key equality comparisons, allocation and link updates. A long string or large integer can make hashing or comparison depend on its encoded length, so counting record visits alone does not establish constant running time.Under simple uniform hashing, each distinct key's bucket index is an independent random variable with a uniform distribution on a finite set of indices. The keys and query are fixed independently of this random assignment. Then lookup has expected time complexity, where is the hash table load factor; it is expected when stays bounded. These are model assumptions, not an automatic consequence of using a deterministic hash function.[ref]
Lecture 8: Hashing I. 6.006 Introduction to Algorithms, Fall 2011 by Massachusetts Institute of Technology, 6.006 course staff; pp. 4–5, chaining worst case and simple uniform hashing assumption
Here is the short calculation. For an absent query and stored keys , let be the indicator random variable of . The number of records inspected has expected valueby linearity of expectation. For a present fixed query, scanning its entire chain inspects at most itself plus all other colliding records, with expected value ; stopping at the match only decreases this upper bound. Adding constant hashing and bucket access yields expected for either query. The argument needs pairwise collision bounds; it does not require all the indicator random variables to be independent. A universal hash family can provide a related bound by random choice of a hash function for a fixed key set, as explained in that article.
The worst case remains , and an absent query takes comparisons if all records occupy its hash bucket. For instance, all collide under . Separate chaining remains correct, but a sequence of insertions that each checks the existing chain can take quadratic total work. Amortized analysis of resizing does not remove this search cost, and an adaptive choice of keys based on a known hash function is outside the stated simple uniform hashing model.
Simple random walk Created 2026-09-24 Updated 2026-10-14
A simple random walk on a locally finite graph chooses each vertex in the current vertex's graph neighbourhood with equal probability, assuming there is at least one available neighbour. On the integer line there are two choices, one unit left or right, so each has probability . This is the one-dimensional simple symmetric random walk. Terminology varies: a simple random walk on the integer line is also often allowed unequal step probabilities, with “symmetric” specifying the fair case.
For a simple symmetric random walk starting at , each increment has the Rademacher distribution, with expected value zero and variance one. Thus linearity of expectation and the variance of a sum of independent random variables giveFor example, after steps the expected value is zero, the variance is , and the standard deviation is . Doubling the typical distance scale takes four times as many steps. This square-root growth explains why a random walk can wander far even when there is no preferred direction.
Four fair and two biased unit-step random-walk paths, with their expected values and bands one standard deviation wide on either side.
Details. The colored curves are illustrative sample paths, not averages over all possible walks. The dashed curves show the expected values; the shaded bands show the expected value plus or minus one standard deviation at each time. These bands are a scale of spread, not confidence intervals, and give no guarantee that an entire sample path stays inside them.
Zero expected value does not make this a stationary process: its one-time probability distribution changes with , as the increasing variance already proves. It does have stationary increments and independent increments, because blocks of fresh steps have the same probability distribution at any starting time and disjoint blocks use independent random variables. This separation of position from increment is one reason the random walk is such a useful first example of a stochastic process.
Simple random walk on the integer line Created 2026-09-24 Updated 2026-10-14
A simple random walk on the integer line starts at and takes independent and identically distributed random variables as unit increments, withThe simple symmetric random walk has ; a biased random walk has . At or every step is deterministic, giving or respectively.
Count the rightward steps using indicator functions:Thus the endpoint is a shifted and rescaled random variable with a binomial distribution. For a worked example, at the endpoint means exactly three rightward steps and one leftward step. There are possible orders, so its probability is , equal to in the fair case. More generally the binomial coefficient counts the orders, giving the exact probability mass functionIn particular, the endpoint must have the same integer parity as : . For the support of a probability distribution is exactly . At only even integers in are reachable; at only odd integers are reachable. This restriction is easy to lose if one draws a smooth curve instead of discrete probability masses. The fair-walk counting argument is also presented in Grinstead and Snell, Introduction to Probability, section 12.1.
Exact binomial endpoint probabilities for fair walks of 20 and 80 steps and a biased walk of 80 steps; only even endpoints are possible.
Details. The endpoint identity and the binomial distribution give the expected value, variance, and standard deviationA short direct check gives and , hence . Then apply linearity of expectation and the variance of a sum of independent random variables.
The absolute width and the width relative to the number of steps answer different questions. In the fair case the standard deviation grows from to , while shrinks from to . At , the standard deviation is , or times the fair width, and . Centering and dividing by each walk's own standard deviation gives, for ,This rescaling equalizes the variances, not the entire probability distributions. The spacing between reachable endpoints is two, so its rescaled spacing is : approximately for the fair -step walk, for the fair -step walk, and for the biased -step walk.
The positions themselves are dependent. For , , where the second term is independent of . Their covariance is thereforeFor example, in the fair case allows or , whereas allows or . Nevertheless the position process is a Markov chain: given its current position, the next fresh increment has no dependence on its past. Its transition probabilities areThese transition probabilities are the same at every time and every integer position: shifting both and by the same integer leaves unchanged. Thus the Markov property and independent increments coexist with dependent positions.





