Strictly proper scoring rule Created 2026-10-07 Updated 2026-10-14
A strictly proper scoring rule is a proper scoring rule for which the true probability distribution is the unique optimizer of the expected value. Equivalently, any different admissible probability distribution gives strictly larger expected loss. For probability density functions, two densities that agree almost everywhere represent the same probability distribution. Ordinary propriety permits ties: the constant reward is a proper scoring rule, because every report ties, but it is not a strictly proper scoring rule and elicits no information.
Strict propriety concerns expectation, not guaranteed success on each observation or each realized dataset. With the stipulated Bernoulli distribution , suppose the outcome is . The honest report then has Brier score loss , while has loss . The logarithmic loss similarly gives losses and . The honest forecast loses this comparison on that draw, which occurs with probability in the stipulated statistical model.
Figure 1.
Individual outcomes can favor a different report even though the expected Brier score and logarithmic loss are uniquely minimized at
. Details.
The loss curves favor , and the loss curves favor . Their weighted average, with weights and , favors . Choosing a report after observing would use information unavailable at forecast time. Even an average loss on a finite dataset can rank an honest forecast below a competing forecast through outcome variation; strict propriety guarantees the ranking in expected value under the specified information and true predictive distribution.
Strong law of large numbers Created 2026-09-24 Updated 2026-10-14
Let be real independent and identically distributed random variables with , and let be their expected value. Their sample mean satisfies
This is almost sure convergence: with probability one, an entire infinite sequence of observations has averages that eventually stay within every prescribed positive tolerance of . The variance need not be finite. The strong law of large numbers implies the weak law of large numbers, which asserts convergence in probability at each increasing sample size.
For independent trials with a Bernoulli distribution of parameter , is one for success and zero for failure. Then is the observed success proportion, and the strong law of large numbers says almost surely. This turns a theoretical probability into a quantity one can estimate by repetition. It does not say that each new trial improves the estimate: a long string of failures can move the proportion away from before later observations bring it back.
Figure 1.
Bernoulli averages approach their mean: sample paths and exact error probabilities
. The left panel shows four reproducibly simulated sequences of independent Bernoulli trials with success probability . The right panel calculates from the exact binomial distribution and compares it with the bound from the Chebyshev inequality. Small irregularities reflect integer-valued counts. The paths illustrate almost sure convergence; a finite simulation cannot prove it. Details.
The strong law of large numbers gives the limiting location of the sample mean, while the central limit theorem describes its fluctuations when the variance is finite and positive. The two assertions can hold together:
The first error shrinks to zero; the second has been enlarged enough to retain a nontrivial probability distribution. The second assertion is convergence in distribution, not a claim that the enlarged error along one sequence approaches a particular normal random variable.
The assumptions make a practical difference. If every observation repeats the same nonconstant Bernoulli random variable, , then for all : identical probability distributions alone do not make repeated observations informative. For independent observations with a Standard Cauchy distribution, the Cauchy sample mean has an unchanged distribution, so it does not concentrate around a constant. Its expected value is undefined, and ; neither example contradicts the stated strong law of large numbers.
A brief proof outline in the case of finite variance uses the Kolmogorov maximal inequality for the centered partial sums. The probability that their maximum up to exceeds is at most . These bounds are summable, so the Borel-Cantelli lemma controls all sufficiently large such blocks and hence every intermediate sample size. The general case requires truncation to control large observations; the argument using finite variance alone does not prove that full version.
A function into a Banach space is strongly measurable if, outside a null set, it is a pointwise norm limit of measurable simple functions. A measurable function with values in a separable Banach space is strongly measurable: approximate values by a countable collection of balls of shrinking radii and then truncate the resulting countably valued approximants to simple functions. A strongly measurable function is Bochner integral integrable exactly when its norm is integrable. Finite second moments under a probability measure therefore imply existence of its Bochner integral expected value by the Cauchy-Schwarz inequality.
At zero field and with spin inversion symmetry in its boundary conditions, a finite Ising model pairs every configuration with its spin-reversed configuration. Their equal Boltzmann weights force the expected value of its spin magnetization to vanish. Below a model's ordering critical temperature, an arbitrarily small positive field can instead select the positive spontaneous magnetization if size is taken to infinity first:
The distinction is the order of limits and the stated symmetric boundary conditions; plus boundary conditions already break spin inversion symmetry. A zero-field symmetric mixture of the two ordered states can retain zero mean spin magnetization even in the thermodynamic limit. Also, in a finite system is not by itself proof of spontaneous symmetry breaking.[ref]
Synaptic vesicle 2026-10-14
A synaptic vesicle is a small membrane-bounded compartment in a presynaptic terminal that stores neurotransmitter for release. Fusion with the cell membrane exposes its contents to the synaptic cleft; the packet supplied by one synaptic vesicle is a quantum of release. Quantal neurotransmitter release refers to these discrete packets, rather than the release of one molecule at a time. The postsynaptic response to a packet can vary with its contents, receptors and recording conditions. See Purves and colleagues: vesicles as the source of transmitter quanta.
The useful cycle is filling, preparation for fusion, calcium-triggered exocytosis, and recovery of membrane and proteins through endocytosis for reuse. Different recovery routes occur; recycling need not preserve a particular synaptic vesicle as an identifiable container throughout. Limited availability of prepared synaptic vesicles can reduce release during repeated stimulation, while changes in synaptic release probability can also change successive responses. See Basic Neurochemistry: secretion machinery and vesicle reuse and 3.1 Synapses.
For an illustrative calculation, assume four release sites, each with one prepared synaptic vesicle, each releasing with synaptic release probability during one presynaptic action potential. Assume independent Bernoulli trials and at most one release per site. The released packet count has a binomial distribution:
The probability of no release is ; at least one release has probability . An expected value of one packet does not mean that every action potential releases one packet. If each packet contributes a stipulated conductance increment and increments add, the expected value of that increment is , with zero increment on about of trials. These assumptions are a teaching model: shared calcium fluctuations, unequal sites, depletion and nonlinear receptor responses can invalidate them.
Tail probability Created 2026-10-03 Updated 2026-10-14
For a real random variable , an upper-tail probability has the form and a lower-tail probability has the form . Each is the probability of an event lying beyond a chosen threshold in one direction. A tail probability need not be small: the threshold determines how much of the probability distribution it includes. A concentration inequality bounds such probabilities, often for deviations from an expected value.
Endpoint conventions matter. Write for the cumulative distribution function and . The complement rule of probability gives four distinct formulas:
The strict upper-tail convention is the survival function . Its value differs from the inclusive upper tail probability by exactly the probability mass at the threshold:
A singleton with positive probability mass is an atom of a measure. Thus excludes an atom of a measure at , whereas includes it. These endpoint identities follow directly from the cumulative distribution function properties in Kyle Siegrist, Distribution and Quantile Functions: endpoint probabilities and right tails.
As the threshold increases, both upper tail probabilities are nonincreasing functions. The strict survival function is a right-continuous function; the inclusive upper tail probability takes the upper endpoint value at a downward jump, equal to the left limit of the survival function. Both tend to zero as for a real random variable, but the speed of that decay differs across probability distributions. The cumulative distribution function properties and their proof from continuity of probability measures are given in MIT 6.436J, Lecture 4: random variables and cumulative distribution function properties.
For a fair six-sided die , the discrete uniform distribution assigns probability mass to each of . At threshold four,
Between four and five, both upper tail probabilities equal . At the largest possible outcome the distinction is especially visible: , but . The diagram shows exact probabilities for this mathematical model; its dots specify which endpoints belong to each step function.
Figure 1.
Exact fair-die probabilities: including or excluding a threshold changes the tail by one atom
. Details.
If a probability distribution has a probability density function , then every singleton has zero probability mass and
For an exponential distribution of rate two, for and zero otherwise, so for and for . Hence . This tail probability is an integral of the probability density function, not its height . A mixed probability distribution retains endpoint jumps even with a smooth component: give zero probability mass and choose this exponential distribution with probability . Then , , and for .
A two-sided tail probability can measure the absolute value of displacement from a specified center . For , the two extreme events are disjoint events, so
For strict displacement , replace the last expression by . The condition prevents overlap; at adding inclusive tails double-counts any probability mass at . For a uniform distribution on , take and : the two tail probabilities are each and their sum is . Equal tails follow from this symmetric probability distribution, not from the phrase two-sided alone; for a uniform distribution on with , the two tail probabilities are zero and .
The tail-sum formula for expectation turns a whole curve of tail probabilities into an expected value. For any nonnegative random variable , without assuming a probability density function or a finite expected value,
For a proof outline, write pointwise and exchange the expected value and Lebesgue integral using Tonelli theorem; is an indicator function. The inclusive version has the same Lebesgue integral: positive probability masses occur at only a countable set of thresholds, which has Lebesgue measure zero. For a nonnegative integer-valued random variable , the tail-sum formula instead gives . The integer hypothesis is essential for these sums. The integral identity, its indicator function proof, and the integer version appear in Kyle Siegrist, Additional Properties of Expected Value: integration and summation of right tails.
For the fair die, this tail-sum formula yields , equal to the direct weighted arithmetic mean . For the rate-two exponential distribution, the area under the survival function is , its expected value. The first panel below shades an integral of the probability density function for a single threshold; the second shades an integral of the survival function across all nonnegative thresholds. These are analytical illustrations, with the displayed curves truncated at three; their exact total areas use an infinite integration range.
Figure 2.
Exponential model: one tail probability is density area, while expectation is total survival area
. Details.
Nonnegativity is essential to interpreting the upper tail probability area as : for a general real random variable that area equals the expected value of its positive part. An integrable random variable satisfies , subtracting the expected value of its negative part; infinite terms cannot be canceled. Even when is finite almost surely, its expected value can be infinite. For the shape-one Pareto distribution on , on and for , so the area is . The finite versus infinite expected value distinction is illustrated in Kyle Siegrist, Definitions and Basic Properties: exponential and Pareto expected values.
Why are tail probabilities useful? They answer threshold questions directly, and a concentration inequality can bound them without knowing the full probability distribution. For nonnegative with finite expected value, Markov inequality gives for : a large threshold relative to the expected value forces rarity. The quantile function reverses the question by choosing a threshold for a prescribed cumulative level. If with , then ; an atom of a measure may prevent equality. For the die, , with strict upper tail probability zero and inclusive upper tail probability .
A tail probability describes rarity under a specified probability distribution; it is not automatically a conditional probability after extra information. For with ,
because the threshold events are nested. Thus the die has but . Conditioning changes the sample space and renormalizes its probability measure; the same extreme outcomes can become common in that restricted population.
Tail-sum formula Created 2026-09-24 Updated 2026-10-18
The tail-sum formula expresses an expected value as the area under a survival function. For every nonnegative random variable ,
This Lebesgue integral and the expected value may both be ; a probability density function is unnecessary. For a random variable taking values in the nonnegative integers, the corresponding sum is
The events and coincide because takes integer values. Thus the strict sum starts at zero, while the inclusive sum starts at one.[ref]
For an original example, let the probability mass function of be
Its expected value computed directly is . The strict tail probabilities at thresholds are : their sum is again . There is no probability mass at two, yet the event still has probability . The survival function has heights on the successive intervals ; each interval has length one, so its integral is the same sum.
Figure 1.
The same expected value from probability masses and survival columns
. The exact constructed probability mass function assigns to . The contribution appears in one survival column, while the contribution appears in three. Filled endpoints give the strict survival function at its jumps; open endpoints show the preceding height. The shaded area is . Details.
The indexing matters numerically. Starting the inclusive sum at zero adds , giving for this example. Starting the strict sum at one omits , giving . For a general nonnegative random variable, replacing with inside the integral is harmless: the difference is , nonzero at only a countable set of thresholds. For each positive integer , at most thresholds can have probability mass at least ; their countable union contains all thresholds with positive probability mass and has Lebesgue measure zero. This reasoning concerns an integral, not equality of the two survival function conventions at every point.
For a continuous original example, take the uniform distribution on the interval . Its survival function is for and zero for . The area is a triangle of base four and height one:
Directly integrating against the probability density function gives the same expected value, . The integral of the probability density function alone is one; it is the survival function whose area is the expected value.
Figure 2.
A uniform density has total probability one and survival area two
. Exact curves for the uniform distribution on . The probability density function has integral one, while the survival function has integral two. Splitting the survival area at gives before the cap and beyond it, the expected values of and the positive part , respectively. Details.
The proof counts layers of each outcome. For a realized value , the indicator function equals one over an interval of length , so
Tonelli theorem permits the interchange because the indicator function is nonnegative; no finite expected value is assumed. Similarly, an outcome contributes one indicator random variable for each of the events that occur:
Again Tonelli theorem applies to the nonnegative sum, or equivalently the monotone convergence theorem applies to its partial sums. This exchanges counting layers within each outcome with averaging across outcomes. The existing integer-valued hypothesis in the tail-sum formula explains what changes when the values are not integers; the tail integral formula for moments weights the layers to obtain higher moments.
For an integrable random variable that can have either sign, apply the nonnegative formula separately to its positive part and negative part :
Integrability makes both integrals finite. More generally the difference defines an extended expected value when at least one is finite; if both are infinite, subtraction does not define an expected value.[ref]
Tail probabilities are often easier to measure or bound than a whole probability density function. For a device's survival time, the survival function records the probability of still operating at each threshold; its integral gives the mean operating time. A finite observation window naturally gives the expected value of a capped nonnegative random variable. For a mathematical bound, for all implies . The tail-sum formula turns control of a whole family of events into control of an average.
Tangent of a uniform angle has a Cauchy distribution Created 2026-10-18 Updated 2026-10-20
If and has the continuous uniform distribution on , then has the Cauchy distribution with location zero and scale . The tangent function is strictly increasing on this interval, so its cumulative distribution function is
Differentiating gives the probability density function . The probability of either endpoint is zero, so including endpoints in the uniform interval does not alter this probability distribution. The slow quadratic decay makes the expected value undefined: its positive and negative parts both have infinite integrals. Its second moment is also infinite, so it has no finite standard deviation.
Model an ideal resistor of constant electrical resistance at uniform absolute temperature by its Thévenin equivalent: a zero-mean random voltage source in series with a noiseless . In the classical range its one-sided, open-circuit voltage power spectral density, per hertz, is
where is the Boltzmann constant. Thus a positive-frequency interval of width contributes to the mean square voltage. This is approximately white noise only where the electrical resistance is effectively constant and , with the Planck constant. The ideal constant spectrum cannot describe arbitrarily high frequencies.[ref]
For a real voltage, an equivalent two-sided power spectral density is over signed frequencies. With a real linear time-invariant system of frequency response , the same output mean square is
Angle brackets denote an expected value; for this zero-mean noise the mean square is its variance. Do not integrate over both signs of frequency: that counts the same fluctuations twice. All integrals here use frequency in hertz, not angular frequency . The amplitude spectral density is in , whereas the root mean square voltage requires a specified measurement signal bandwidth.
Let be the number of projectiles actually incident on the foil during an exposure, its foil areal number density, and the probability of detecting an accepted particle. For a sufficiently small detector patch of area at distance from the scattering point, with its normal at angle to the line of sight,
The product is a dimensionless scattering probability in this approximation; is an expected value of counts, so it need not be an integer. The approximation requires a nearly constant differential scattering cross-section and efficiency across the patch and incident beam. For finite acceptance use , including the incident beam's spread if appreciable. A screen facing the foil has .
For an original numerical model, stipulate monoenergetic alpha particles with , , and a gold target with . Use the rounded model constant , where electronvolts and . Thus and . Choose , , , and . For example, facing the foil at gives this small-patch solid angle.
At , and , using barn . The expected signal is counts. At it is four times larger, about counts. Doubling foil areal number density or detector solid angle doubles these expected signals as long as the approximations continue to hold.
Figure 1.
Calculated Rutherford angular cross-section and finite-detector counts
. Original calculation for the stated ideal monoenergetic alpha beam and thin gold foil. The curve is a differential cross-section; the bars are expected counts for the specified small detector solid angle at separate settings. They are not measured 1909 or 1913 data, and do not sum to a total scattering probability. The curve is shown only from 20 to 160 degrees. Details.
An observation also contains backgrounds. For a stable source and matched geometry, an estimator is , where denotes exposure duration. If source activity changes, normalize by incident exposure as well. Background subtraction removes an estimated background; it does not make finite counting fluctuations disappear.
Memory thrashing is repeated costly cycling of memory pages out of and back into computer memory, leaving little useful computation between page faults. Active demand can exceed the available page frames, especially when multiprogramming admits too many operating-system processes. A high page fault rate during initial loading alone does not establish memory thrashing. [ref]
Consider one operating-system process making the 20 memory references : five repetitions of four memory pages. Start with an empty resident set, use demand paging and exact LRU replacement, and assume no prefetching or competing operating-system processes. A hit finds the requested memory page resident; a miss requires a nonresident page fault. With three page frames, references 1–3 load pages 1–3. Reference 4 evicts page 1, reference 5 reloads page 1 and evicts page 2, and the cycle continues: every return finds its memory page evicted. With four page frames, only the first four references miss; all four memory pages then remain resident.
Available page framesMisses in all 20 memory referencesOverall page fault rateMisses in references 5–20Page fault rate in references 5–20320100%16100%4420%00%
The four initial misses with four page frames are compulsory first-use misses from the empty resident set. They are startup work; the three-page frame case keeps faulting after startup. This makes the difference between memory thrashing and loading a new working set visible. For the three-page frame trace, each reused memory page has three distinct intervening memory pages, so LRU replacement has already evicted it before its next use.
Figure 1.
The same 20 memory references under LRU replacement: three page frames miss every time, four miss only during startup
. Details.
To quantify the cost, assume each hit takes 100 nanoseconds and each miss incurs an additional 1 millisecond of page fault service, followed by the same 100-nanosecond memory access. These are toy latencies, not measurements. With miss probability , the expected value of access time is
The service cost is additional: it does not include the final 100 nanoseconds. Thus gives and gives , approximately a fivefold ratio over the complete trace. Over references 5–20, the corresponding ratio is : repeated page faults dominate the useful accesses. The expected value model follows the additive miss-penalty convention of OSTEP chapter 22, section 22.1. [ref]
Not every page fault performs disk input-output: demand paging can allocate zero-filled memory pages, and copy-on-write can copy an existing page frame. The toy service cost models expensive misses; the miss trace alone establishes residency behavior, while memory thrashing additionally concerns repeated costly cycling. [ref]
Possible remedies are more resident page frames, less multiprogramming, or reorganizing accesses to improve temporal locality. For this toy sequence, one extra page frame suffices under LRU replacement; that is not a universal threshold or policy. Page replacement alone cannot make every simultaneously needed memory page fit.
// Target: computer-science.bigb
Suppose annual claim counts are independent with geometric distribution mass , . In a no claims discount system with discounts , a claim returns either of the first two levels to zero; at the top, one claim sends the level to and two or more send it to zero. The transition matrix is . Solving gives the displayed stationary distribution. With full premium , the stationary expected value of the premium is . Independent annual sampling is essential; a persistent random intensity does not justify this marginal transition matrix.
On with Lebesgue probability and the usual dyadic filtration, each displayed support is a dyadic atom. Its next split has one child of doubled height and one of zero height, preserving the conditional average. Hence the sequence is a martingale from time one. To extend it to the trivial time-zero sigma-algebra one must set : applying the same displayed formula at zero instead gives , which is not adapted and has the wrong expected value. The repaired martingale tends to zero almost surely while its expected values remain one, so it is not uniformly integrable.
For an almost surely finite total progeny of a branching process, let be the offspring probability generating function and that of the total, including one ancestor. Conditional on children, the total is one plus the sum of independent descendant totals. Independence and the law of total expectation give for , including the empty-product case . If the offspring expected value is , differentiation for gives . The denominator is positive because . Letting and using the monotone convergence theorem on the derivative series proves , establishing finiteness rather than assuming it. For Poisson branching process offspring, .
For independent Poisson process arrivals of geometric translational bursts with mean , arrival rate , and unit protein removal rate , the stationary mean is . Starting with zero molecules, the expected value is . The variance satisfies . An integrating factor gives the displayed expression, and its stationary limit is . Concentration fluctuations require a specified cell volume to convert concentrations into molecule counts.
Consider one chosen model at a fixed total energy, with two open scattering channels, no absorption and no omitted outgoing modes. Channel 1 leaves the target unchanged; channel 2 excites it. Let and . For the same projectile mass, conservation of energy then requires a target excitation . These are hypothetical parameters, not a fit to a particular atom. Each incoming or outgoing mode is normalized to unit probability current; the outgoing modes will have the angular profiles specified below.
Choose the S-matrix
This phenomenological unitary matrix specifies the channel conversion; it is not derived here from an interaction potential energy. For incidence in channel 1, its first column gives the outgoing flux amplitudes
To recover ordinary wave coefficients relative to an incident coefficient one, undo the velocity normalization:
Thus is allowed: channel 2 travels more slowly. Its outgoing probability current fraction is , not . Renormalizing the unweighted squares would give and , the wrong probabilities. More generally the ordinary coefficient map is , with ; it obeys . The conserved quantity is the squared coefficient norm weighted by velocity, which represents probability current.
Now specify an original angular realization of these two outgoing modes, with scattering angle and no azimuth dependence:
The first normalization follows from , where . These angular profiles define the modes of this model; they are not asserted to follow from Born's approximation or an atomic potential energy. The joint directional probability densities are
Figure 1.
Complex angular amplitudes become channel-resolved scattering intensity
. Original two-channel model. Left: arrows in the complex plane show , the angular factor of . Right: the joint directional probability densities , per solid angle, retain the channel flux weights and . The shaded interval is the cone ; its probability requires the measure , not the area under a curve against degrees. These are chosen amplitudes, not historical measurements. Source. Details.
Suppose a detector accepts the cone , with full azimuth and unit efficiency. Its angular fractions within the channels are
Multiplying the angular fractions by the outgoing flux weights gives
For independent incident events, the expected values are therefore and accepted detections, compared with and over all directions. The remaining expected events fall outside the cone. The chosen incident number makes the expectations integers; actual counts fluctuate. Conditional on a detection inside the cone, the channel probabilities are and , rather than and . Angular acceptance changes the observed channel proportions even with equal detector efficiencies.
Figure 2.
Velocity weighting and angular acceptance determine expected channel counts
. Original model with incident events. Left: raw squared wave coefficients and become conserved outgoing flux fractions and after multiplying by . Right: all-direction expectations and become and inside the cone. Bars show calculated expected values, not a generated sample or experimental counts. Source. Details.
The final atomic states are orthogonal, so ignoring the atomic readout adds the two channel intensities; it does not produce . Coherent paths that reach the same final state can instead interfere, as the existing probability amplitude and quantum superposition examples demonstrate. The distinction is physical distinguishability of final states, not whether the detector operator chooses to display their labels. Flux normalization and angular integration make the scattering interpretation useful: they turn complex outgoing waves into conserved event weights and then into the statistics of a specified detector.
The normalized vacuum state annihilated by and , for positive with . Since , it can also be written . Acting with either annihilation operator makes the two terms cancel. Its original occupation numbers each have expected value and its pair amplitude is . It has even fermion parity although the original total number operator is not sharp. For the two-orbital fermionic pairing Hamiltonian, it is the unique ground state of energy .
Let be the weak typical set of a finite alphabet independent and identically distributed source with Shannon entropy , and let be any sets with . The asymptotic equipartition property and union bound give . Every word in the intersection has probability mass between and . Summing these bounds gives
For every fixed , the first probability eventually exceeds , proving the title bound. A sample mean within a fixed positive tolerance of the expected value supplies by the strong law of large numbers.
The uncentered empirical second-moment matrix averages the outer products of the observations. Its expected value is . It is an unbiased estimator of the covariance matrix when the population expected value is known to be zero. It differs from a sample covariance matrix centered at the sample mean.
Uniform distribution with an atom at zero Created 2026-10-18 Updated 2026-10-20
For and , this mixed probability distribution has probability mass at zero and ordinary probability density function on . That ordinary density integrates to , not to one; conditioned on , the law is the uniform distribution on . Its cumulative distribution function is zero for , for , and one for .
For each positive integer , integration gives . Thus the expected value is and the variance is . The nonzero atom contributes to normalization even though it contributes zero to positive moments. At , conditioning on positive delay is undefined because the conditioning event has probability zero.