List-decoding Fano inequality 2026-10-06
If a uniform index has possibilities and every proposed success neighbourhood contains at most possibilities, then any estimator has failure probability at least the displayed expression. Condition on its success indicator: conditional entropy is at most . Subtract this from to bound the mutual information and rearrange. A metric Hamming ball gives approximate recovery rather than exact index decoding.
Load factor of a hash table Created 2026-10-13 Updated 2026-10-18
The number of stored records divided by the number of hash buckets. For separate chaining, this is the mean chain length over all hash buckets, including empty ones.
Consider two original separate chaining placements using the deterministic hash function on nonnegative integer keys. With , keys give the balanced chain lengths . Instead, keys all select hash bucket , giving . Both store records, so both have . The maximum chain lengths are and . These are constructed placements, not measured workloads.
The same hash table load factor permits balanced and concentrated hash bucket occupancies
. Both original examples use eight hash buckets and twelve distinct integer keys. Every outlined column counts towards , including empty ones; each colored square represents one record. The dashed line marks the arithmetic mean , not the maximum chain length. Details.For an absent query, separate chaining inspects every record in the selected hash bucket. Key selects hash bucket in both examples but is stored in neither, so it takes two key equality comparisons in the balanced placement and twelve in the concentrated placement. Key selects hash bucket and instead takes one comparison in the balanced placement and zero in the concentrated placement. If an absent query's bucket index has the uniform distribution on a finite set of eight indices, its mean comparison count is in either fixed placement. That average does not limit the cost of a particular query.
For successful lookups, suppose each of the twelve stored keys is queried with equal probability, with fixed chain order and no rearrangement on access. A chain of length contributes comparison counts , irrespective of its order. ThusThe balanced placement gives comparisons; the concentrated placement gives . Even this average depends on more than : concentration appears through the squared chain lengths. These exact counts exclude computing the hash function, locating the hash bucket, and allocating records. Elapsed lookup time also depends on the cost of key equality, key length, and memory access latency; alone specifies none of them.
The expected time complexity bound for separate chaining needs a probabilistic assumption. Fix distinct stored keys and a query independently of the random choice of hash function, and require for every unequal pair, where is a constant independent of . With constant-cost hashing, array access and key equality, this suffices. The authored chaining analysis cited here uses this pairwise-collision argument; its multiplicative construction has , rather than the of simple uniform hashing.[ref]
5.1 ChainedHashTable: Hashing with Chaining. Open Data Structures: An Introduction by Pat Morin; section 5.1.1, Lemma 5.1 and the indicator-variable proof of Lemma 5.2
For the proof outline, let be the indicator random variable that stored key shares an absent query 's hash bucket. By linearity of expectation,A present query's full chain contains itself plus at most other records, giving . Stopping at the match can only improve this bound. Full independence is unnecessary for this calculation, but fixed known hash functions and keys chosen adaptively to collide are outside the stated assumption. Keeping bounded then controls expected lookup work, while the concentrated example still explains why it gives no worst-case guarantee.
For the balanced twelve-record map,
put(3,new_value) replaces a value, so and remain unchanged. An absent key such as instead adds a thirteenth record, giving if capacity stays fixed. Under a policy that stores every repeated-key record separately, another record with key would also increase ; all equal-key records necessarily share a hash bucket when the hash function respects key equality. State the map policy before interpreting the count.The appeal of the hash table load factor is that one number helps tune a space-versus-search tradeoff: more hash buckets require more array storage and can reduce expected search work under the stated assumptions. The actual chain lengths and query choices still matter.
In open addressing, counts record slots and counts live records, so . A policy retaining at least one genuinely empty slot requires ; being below one is not itself a speed guarantee. For deletion using a tombstone in open addressing, also track , the number of marked slots: counts slots that cannot terminate an unsuccessful probe. For example, gives live occupancy but nonempty-or-marked occupancy . These ratios answer different questions.[ref]
5.2 LinearHashTable: Linear Probing. Open Data Structures: An Introduction by Pat Morin; section 5.2, three entry types, counters n and q, and find stopping at nil
Local lemma Ramsey lower bound 2026-10-14
In an independent random two-colouring of , a given -vertex set is monochromatic with probability . Events on sets intersecting in at most one vertex use disjoint edges. Thus dependency degree is at most . The Lovász local lemma gives a colouring avoiding all monochromatic when . For the expression is , by Stirling's approximation. Taking tending sufficiently slowly to proves .
Logarithmic loss 2026-10-14
The loss function assigning to a predicted probability distribution and observed outcome . It is the negative logarithmic scoring rule; its expected value under the true probability distribution is cross-entropy. Predicting zero probability for an observed outcome gives infinite loss.
Logarithmic scoring rule Created 2026-10-07 Updated 2026-10-14
The logarithmic scoring rule rewards the natural logarithm of the reported probability mass or probability density at the observed outcome. The title uses the reward convention. For a binary report , its negative is the logarithmic loss:For this is ; for it is . At an endpoint, use these outcome-specific forms: assigning zero probability to the outcome incurs infinite loss, while assigning it probability one gives zero. The natural logarithm therefore makes a confidently wrong binary forecast far more costly than the bounded Brier score does.
Under the stipulated Bernoulli distribution with , the expected logarithmic loss isIts unique minimum is at , where the expected loss is approximately . At it is approximately . More generally, the excess expected logarithmic loss over truthful reporting is the Kullback-Leibler divergence in the order true law first, reported law second:For , this Kullback-Leibler divergence is nonnegative and vanishes only at , explaining why the logarithmic scoring rule is a strictly proper scoring rule. The same Kullback-Leibler divergence identity applies to probability density functions relative to a common measure when the truthful expected logarithmic loss is finite and the comparison expected value is well defined.[ref]
Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others. Section 3.1, Example 3, and Section 4.1
For sequential forecasts, the existing prequential log score identity explains how logarithmic rewards add over observations.Assume that each physical qubit undergoes an independent quantum bit-flip channel with probability during one storage interval, and that encoding, error syndrome extraction, recovery and decoding are all ideal. Use the three-qubit bit-flip syndrome table to apply the indicated minimum-weight recovery. Zero or one bit flips are corrected. Two bit flips have the same error syndrome as the complementary one-site error: for example, gives , so recovery leaves . Three bit flips give and also leave . This exchanges and , and therefore acts as a logical Pauli X gate after decoding.
The number of physical bit flips has a binomial distribution with three trials, hence the logical-error probability isFor a logical density operator , the decoded quantum channel is exactly . Here logical failure means a nontrivial logical Pauli X gate error; a particular input such as can be invariant under that error. At , exactly two flips have probability and three flips have probability , giving . The block succeeds with probability compared with for an unencoded qubit. At , , illustrating the quadratic suppression of rare independent bit flips.
Factoring the improvement givesThus one level reduces logical-error probability for , is equal at , and makes it worse for under this same decoder. The suppression arises because a logical error requires at least two physical bit flips. With independent disjoint inner blocks, the same noise model and ideal correction at every level, a concatenated quantum error-correcting code has , with . At , two levels use nine data qubits and give .
This calculation describes ideal correction of stored information with the specified quantum bit-flip channel. Repeated correction in time additionally requires a fresh specified noise model for each interval; imperfect quantum gates, faulty measurements and correlated errors change the calculation. The timed bit flip during repetition-code syndrome extraction and exact encoded-state failure with a faulty parity check show concretely why this storage calculation alone is not a fault-tolerance result.
The number of monomials in variables with each exponent at most two and total polynomial degree at most satisfiesFor each exponent vector counted on the left, . Summing and then allowing all gives . The base has cube , so it is strictly less than three. This elementary weighted count gives the exponential saving in the polynomial method in combinatorics without an unstated probability estimate.
Lp norm Created 2026-09-24 Updated 2026-10-14
An Lp norm measures the size of a measurable function by averaging a power of its absolute value with respect to a specified measure. For ,when the integral is finite. The exponent controls how strongly large values influence the result. The Lp space identifies functions equal almost everywhere, so this becomes a norm on the corresponding equivalence classes. For , use the essential supremum of , as defined by the L-infinity norm. [ref]
Lp spaces (Chapter 7 of Measure Theory). Hunter, Measure Theory, Definitions 7.1 and 7.4: Lp spaces and norms
.For a two-coordinate vector , treat the coordinates as a function on two inputs with counting measure. Then , , and . The middle case is the Euclidean norm; the first adds the coordinate magnitudes, and the last takes their largest value. These are different answers to different questions about size, all satisfying the triangle inequality.
If instead each input has probability , the same values give and , while remains unchanged. Thus the measure is not an optional label. On a probability space, the finite Lp norms are nondecreasing in whenever they are finite, by Hölder's inequality; the unnormalized two-coordinate example above decreases instead. Statements comparing different exponents must specify their measure convention.
Why are these sizes useful? If is an error function, its L1 norm measures aggregate absolute error, its L2 norm measures the square root of aggregate quadratic loss, and its L-infinity norm bounds error almost everywhere. Concentrating a large error on a set of small positive measure can give a modest finite Lp norm but a large L-infinity norm. The three-point example and diagram in L-infinity norm show this explicitly, including why a null set behaves differently from a rare event.
Lüders rule Created 2026-09-28 Updated 2026-10-14
The Lüders rule specifies the ideal quantum instrument for a projective measurement. Conditional on an outcome with nonzero probability, it givesfor a density operator and a pure state, respectively. The second formula applies only to a pure state input. If the result is discarded, the output is the nonselective projective measurement .
The crucial feature for a degenerate eigenvalue is preservation of quantum coherence within its eigenspace. For a qutrit with orthonormal basis , considerOn , outcome zero has probability one and the Lüders rule leaves unchanged. Measuring all three rank-one projectors and then forgetting whether the label was zero or one instead givesBoth procedures always report the coarse value zero, but a subsequent test with projector succeeds with probability one for the first and one half for the second. Resolving extra information destroyed coherence. This explains why a quantum nondemolition measurement of a degenerate quantity must specify what information the apparatus actually records.
Gerhart Lüders proposed this state-change rule in 1951, explicitly discussing degenerate quantities and distinguishing selected subensembles from the aggregate measured ensemble. Kirkpatrick's translation was first posted in 2004; equations (7) and (8) give the unnormalized selective update and the nonselective update. [ref]
https://arxiv.org/abs/quant-ph/0403007. Lüders, 1951, translated by Kirkpatrick, 2004
.Repeatability follows briefly from : after outcome , . This assumes a second application of the same ideal instrument without intervening dynamics. An instrument with the same effects can rotate or reset the output and need not satisfy this conclusion; see POVM does not determine the post-measurement state.
Luria-Delbrück experiment 2026-10-18
In 1943, Salvador Luria and Max Delbrück grew separate Escherichia coli B cell cultures from small equal inocula, without the challenging bacteriophage. They used nutrient broth or an asparagine–glucose cell culture medium. On nutrient agar plates, they first spread excess bacteriophage and then the bacterial sample centrally, leaving an outer margin. After incubation, resistant biological colonies provided the readout; counts were usually made at 24 and 48 hours. Their control plated repeated samples from one shared culture. Independently grown cultures varied much more than this sampling comparison. That contrast supported resistant lineages arising before challenge: early mutations could produce many resistant descendants before natural selection revealed them.[ref]
Mutations of Bacteria from Virus Sensitivity to Virus Resistance by Salvador Luria and Max Delbrück. pp. 501–504, Material and Methods; Experimental, plating reliability, Table 1 and independent cultures, Table 2
This was evidence about the timing of heritable resistance, not DNA sequencing of the responsible genes. It does not establish uniform mutation rates or exclude environmentally induced mutations; the paper itself discusses physiological dependence and delayed expression.[ref]
Mutations of Bacteria from Virus Sensitivity to Virus Resistance by Salvador Luria and Max Delbrück. pp. 505–507, discussion of excess variation, physiological state, delayed resistance and genetic relatedness
Now construct a deliberately small mathematical model of the timing effect. Start with one sensitive biological cell at generation . Every biological cell divides into two in each of five synchronous rounds, with no death or growth difference. There are biological cells immediately after round , ending with . In each scenario, exactly one newly formed daughter changes to resistance immediately after round ; its descendants inherit resistance without reversion or further mutations. Resistance is expressed immediately, and the final challenge allows every resistant biological cell, and no sensitive one, to form one separate biological colony.
The descendant count obeysAt the moment of the event there is one resistant biological cell; each remaining round doubles its descendants. This proves the formula by repeated multiplication, with terminal case . An event after round 1 gives resistant and 16 sensitive biological cells; an event after round 4 gives resistant and 30 sensitive biological cells. Both scenarios contain one mutation event. A culture with no event gives zero resistant biological colonies under these assumptions. The difference is family size, rather than the number of events.
For comparison, suppose each of the 32 final sensitive biological cells independently acquires resistance only on challenge, with probability and no subsequent multiplication before counting. Then the resistant count has a binomial distribution, with expected value and variance . This alternative mathematical model has no pre-challenge resistant family to amplify. Its independent trials are an assumption of the comparison, not a consequence of using physically distinct biological cells. The clonal scenarios explain how a jackpot can occur; they do not assign probabilities to the event times or estimate a real mutation rate.
// Target: biology.bigb
Lyapunov moment inequality Created 2026-10-06 Updated 2026-10-14
The Lyapunov moment inequality says that higher positive absolute moments give no smaller absolute moment sizes on a probability space. For a real random variable and ,allowing an infinite value. For the left expression is an Lp norm; below one it is a quasi-norm, so the usual triangle inequality terminology needs that distinction. The comparison uses a probability measure of total mass one; arbitrary unnormalized measure spaces need a factor depending on their total mass.
For a nonnegative count, the first moment size is its expected value, while the second is the square root of its second moment. Both ten-trial counts in second moment method have first moment size one. The independent count has second moment size , whereas the shared-outcome count has second moment size . The second moment distinguishes models with the same average because its square increases the influence of rare large counts.
More generally, for with a Bernoulli random variable of parameter ,It rises from one at toward ten as . Replacing by a parameter gives , which decreases toward zero for a fixed positive exponent when , making the rare-event and large-exponent limits different questions. For a scaled Rademacher random variable taking values and two with equal probabilities, every absolute moment size is exactly two: increasing the exponent does not distinguish signs when the absolute value is constant.
Lyapunov moment inequality reveals how higher moments emphasize large outcomes and when equality holds
. Left: the two original ten-trial models from second moment method have the same mean one, but different absolute moment sizes at exponents above one. The shared model has with . Right: equal masses at and two keep almost surely, giving equality for every pair of positive exponents. These are fully specified mathematical probability models. Details.For finite , apply the reversed Jensen's inequality to the strictly concave function and the nonnegative random variable :Taking the positive th root proves the claim. Equality in this finite case holds exactly when is constant almost surely. If the higher moment is infinite, the upper bound is infinite and remains valid. Exponent zero is excluded from the statement; logarithmic or geometric limiting summaries require separately stated conventions, especially when can vanish.
Mach-Zehnder interferometer 2026-10-14
A Mach-Zehnder interferometer splits an incident wave into two arms and recombines them at a second beam splitter. A relative quantum phase changes the interference at its output ports. With symmetric lossless 50:50 splitters, the ports can be calibrated to have detection probabilities and .
// Target: optics.bigb
Macrostate Created 2026-09-24 Updated 2026-10-14
A macrostate groups microstates sharing prescribed coarse data. The grouping depends on what is retained: here retain only total energy. For the two distinguishable two-level systems, the macrostates have state degeneracies . Although two units are not macroscopic in size, they make this loss of information visible: knowing does not tell us which unit is excited.
At inverse temperature , each unit's excited-state probability is . Their independence gives microstate probabilities in the order above. Adding within each macrostate givesThe total expected value of energy is therefore , matching two units each with expected value of energy . Equal state degeneracies would give the wrong answer: macrostate probabilities combine counting with the Boltzmann factor. At very high positive temperature, all four microstates become equally likely, but these three macrostate probabilities approach .
Magnetic entropy Created 2026-10-18 Updated 2026-10-20
Magnetic entropy counts the uncertainty in the occupied magnetic microstates. For independent localized magnetic moments with one-site probabilities ,where counts sites and is the Boltzmann constant. This is the Gibbs entropy for the magnetic degrees of freedom. Equal occupation of two states gives ; certain occupation of one state gives zero. Calling this disorder is useful only if the relevant states and their probabilities are specified. In an interacting material, correlations generally prevent replacing the many-body Gibbs entropy by this sum of independent one-site contributions.
Increasing the magnetic field at fixed positive temperature favors lower magnetic energy levels, reducing magnetic entropy. During ideal isolated demagnetization, both the level separation and temperature fall together, leaving the occupation probabilities unchanged. Thus the independent-spin idealization cools without increasing spin disorder along the isolated leg. When a real crystal's vibrating lattice also cools, its lost entropy can instead be taken up by the magnetic degrees of freedom. The conserved quantity is the total entropy of the thermally isolated stage, including any attached load in thermal equilibrium.
Take a uniformly random permutation of distinguishable objects, of which are marked. Let mean that position is marked. Every object is equally likely at a specified position, so . For distinct positions , their ordered pair is uniform among distinct-object pairs, of which are marked pairs. This proves the second formula. For , the conditional probability is .
For the count of marked objects among the first positions, indicator random variables and linearity of expectation give . These draws are exchangeable but generally dependent; equal marginal probabilities do not imply independence.
Mark and recapture 2026-10-14
Mark and recapture estimates the cardinality of a finite set of individuals when counting everybody is impractical. The useful information is the overlap between observations: with fixed sample sizes, finding many of the same individuals again suggests a smaller population, while finding few suggests a larger one. More elaborate capture-recapture models also estimate survival and changing probabilities of detection.
In a closed-population capture-recapture model, let be the unknown population cardinality. Capture distinct individuals, mark them, release them all, and allow them to mix with the rest. Next take a second sample of distinct individuals using sampling without replacement, and count the individuals carrying a mark. The observed marked fraction estimates the population's marked fraction , giving the Lincoln-Petersen estimatorRelease happens between the two samples; sampling without replacement means that an individual is counted at most once within the second sample. See Richard J. Barker, Open Population Mark-Recapture Models, §1.1.1.
For a constructed teaching example, mark individuals and later capture , including marked individuals. One fifth of the second sample is marked, suggesting that the twenty marked individuals are one fifth of the population:Let and be the sets of individuals in the first and second samples. Their intersection has cardinality five. By the inclusion-exclusion principle, the set union containsThus is an observed lower bound, whereas is the output of an estimator based on the capture-recapture model. These are toy counts, not measurements from a medical or ecological study.
Forty distinct observed individuals: fifteen in the first sample only, five shared, and twenty in the second sample only
. Details. The diagram shows only observed individuals. It does not draw an assumed population of one hundred. The capture-recapture model uses the overlap to infer how many individuals may have escaped both samples; its likelihood function shows why one estimator output cannot describe all that uncertainty.
Markov chain Created 2026-09-24 Updated 2026-10-14
The term Markov chain commonly denotes a Markov process on a finite or countable state space. Time may be discrete, as for a daily record, or continuous, as for a continuous-time Markov chain. Some authors also use “Markov chain” for discrete-time processes on more general state spaces; specifying both time and state space avoids this ambiguity.
For a Discrete-time Markov chain, its initial probability distribution and transition matrix specify the probability of each finite path:This is the Markov factorization, obtained by applying conditional probability repeatedly. See Hao Wu's MIT lecture 1, definition and distribution evolution.
For the constructed sunny/rainy Markov chain below, starting sunny makes the path sunny, rainy, sunny have probability . Starting instead with stationary distribution makes that path's probability . The transition probabilities are identical in both experiments; the initial probability distribution distinguishes them.
Markov process Created 2026-09-24 Updated 2026-10-18
A Markov process is a stochastic process whose present state contains all the information in its observed past needed to predict its future. Precisely, if is the natural filtration generated by observations up to time , then, for and any measurable set of states,The equality is between conditional probabilities and holds almost surely. It expresses the Markov property: the past and future have conditional independence given the present. It does not assert independence of consecutive states. In the sunny/rainy example below, tomorrow's sunny probability is after sunshine but after rain, so knowing today's state matters.
The memory assumption concerns the chosen state space. If an equipment model's failure probability depends on its age, recording only “working” or “failed” may lose relevant information; recording age as part of the state can retain it. Similarly, a dependence on two previous observations can sometimes be represented by using their ordered pair as the current state. Whether such a description fits observations is a modeling question.
Transition Markov kernels describe the conditional distribution of a state at a later time. Successive kernels and the initial probability distribution determine every finite-dimensional distribution, through repeated conditional probability. The time-homogeneous Markov property is an additional assumption: the kernel depends on elapsed time rather than calendar time. The attraction is compression: prediction can proceed from a current state instead of an entire recorded history. Brownian motion and continuous-time Markov chains illustrate that a Markov process need not evolve in discrete time.
Markov's inequality 2026-10-14
For a nonnegative random variable and threshold , Markov's inequality givesThe useful case has finite expected value; an infinite right side gives no finite information. The bound also holds if is nonnegative almost surely. Combining it with the elementary bound one gives . Nonnegativity is essential: negative values could otherwise cancel the contribution of large positive values to the expected value.
The idea is an average budget. Every outcome where contributes at least times its probability to , and the other outcomes contribute nonnegative amounts. Formally, the indicator function gives ; taking the expected value proves the bound. [ref]
https://ocw.mit.edu/courses/18-175-theory-of-probability-spring-2014/887e96172e82a615dac16fb26702d88b_MIT18_175S14_Lecture8.pdf. Sheffield, MIT 18.175, lecture 8: the expectation argument
.For example, let take , , , with respective probabilities , , , . Its expected value is . At , the tail probability is , while the bound is . At , the tail probability is , while the bound is . A different probability distribution with the same expected value, putting masses and at and , attains the bound exactly at .
Markov's inequality is useful when a moment is easy to compute but a full probability distribution is difficult to determine. A nonnegative count with expected value less than one must have some positive probability of being zero, giving the first moment method. Applying the same bound to a squared centered random variable gives Chebyshev's inequality. Applying it to an exponential function of a random variable gives an exponential Markov bound, which can exploit more than a single moment.
Matching pennies 2026-10-14
Both players choose heads or tails without seeing the other's choice. The row player receives when the choices match and when they differ; the column player receives the opposite payoff. Thus matching pennies is a zero-sum game, with ordered payoff pairs:
| Row choice / column choice | ||
|---|---|---|
At the column player improves by switching to ; at the row player improves by switching to ; at the row player improves by switching to ; at the column player improves by switching to . Every pure strategy strategy profile therefore has a profitable unilateral deviation, so none is a Nash equilibrium.
Let be the row player's probability of heads and the column player's probability of heads. Their random choices are independent. Against fixed , the row player's pure-choice expected payoffs areAgainst fixed , the column player's pure-choice expected payoffs areConsequently the best responses, expressed as allowed heads probabilities, areHere means every mixture is a best response, including both pure strategies. If , the row player must choose , which makes the column player's best response , a contradiction. If , the row player must choose , which forces , also a contradiction. Hence ; for the column player to mix at this , the two column payoffs must tie, forcing . The unique Nash equilibrium is thereforeIndeed, for any , and . Holding gives the row player zero for every deviation ; holding gives the column player zero for every deviation . A player is indifferent against the equilibrium mixture, but choosing an arbitrary best response to it need not preserve the opponent's best response. This is why that indifference does not produce additional Nash equilibria.
For a teaching treatment with the same payoff convention, see MIT 17.810, lecture 3, slides 29–38, on mixed strategies and matching pennies.







