Let . The Bernoulli distribution likelihood function is . Multiplication by the beta distribution prior density and Bayes' theorem giveRecognizing the normalized beta distribution,This is Beta-binomial conjugacy; the same update applies whether the data are recorded as the individual Bernoulli trials or their success count.
Let be the head indicator random variable for toss . These are independent and identically distributed random variables with a Bernoulli distribution of parameter , so , , and . Taking makes . The preceding Chebyshev's inequality bound givesThe endpoints are included because this event is exactly .
Let indicate that student stays awake. These indicators have independent Bernoulli distributions with parameter , and . The Poisson distribution has probability generating functionThe probability generating function of an independent random sum now givesRecognizing the Poisson distribution,This is Poisson thinning.
Using the two masses of the Bernoulli distribution,This probability generating function is a polynomial, so it exists for every finite .
For The first is a discrete uniform distribution; the second is a Bernoulli distribution on Boolean values. Their exact mathematical probability mass functions areEach support contains exactly the positive-probability outcomes once, and the probabilities sum to one.
Uniform, require a nonempty integer interval. Compute its size in Long before converting to Double, avoiding integer overflow in u-l+1. For Flip, require , and omit zero-probability outcomes from the reported support.class Uniform(l: Int, u: Int) extends FiniteDist[Int] {
require(l <= u)
private val size = u.toLong - l.toLong + 1L
def getDom: List[Int] = {
val result = List.newBuilder[Int]
var x = l.toLong
while (x <= u.toLong) { result += x.toInt; x += 1L }
result.result()
}
def getProb(x: Int): Double =
if (l <= x && x <= u) 1.0 / size.toDouble else 0.0
}
class Flip(p: Double) extends FiniteDist[Boolean] {
require(0.0 <= p && p <= 1.0)
def getDom: List[Boolean] =
if (p == 0.0) List(false)
else if (p == 1.0) List(true)
else List(true, false)
def getProb(x: Boolean): Double = if (x) p else 1.0-p
}Double approximates these exact values by floating-point arithmetic. Materializing a huge support is limited by available memory; the Long calculation keeps its cardinality and endpoints arithmetically correct.Implement the Bernoulli distribution, taking special care that a zero-mass outcome is excluded from the support of a probability distribution:The precondition excludes NaN and all values outside , including infinities. The two masses are nonnegative and sum mathematically to one. Endpoint domains are singleton lists, while an interior parameter lists both outcomes exactly once. Treating
class Flip(p: Double) extends FiniteDist[Boolean] {
require(!p.isNaN && 0.0 <= p && p <= 1.0)
def getDom: List[Boolean] =
if (p == 0.0) List(false)
else if (p == 1.0) List(true)
else List(true, false)
def getProb(x: Boolean): Double =
if (x) p else 1.0 - p
}getDom as always containing both Booleans would violate its positive-support contract at the endpoints.Summing the two nonzero entries with givesThe remaining probability is , so has the Bernoulli distribution with parameter .
For the other marginal, the two entries at add to , the entry at is , and for the only nonzero entry is already . Thusand has the Poisson distribution with parameter . This also checks normalization of the joint probability mass function by summing the marginal.
Photon-recoil lag correlation Created 2026-10-18 Updated 2026-10-20
A photon-recoil lag correlation asks how often a photon record occurs at a specified time displacement from an electron record. Let be observed separate-channel rates, the rate of detected genuine pairs before a timing cut, and their normalized timing probability density function. For an original dilute, stationary model with observation time , neglecting end effects and detector dead time, the expected pair density isThis counts all pairs of records, rather than at most one detector trigger per electron. It uses independent unrelated detections for the flat term and an added same-event contribution. A normal distribution for timing spread is a chosen approximation, not a claim about the historical instrument. Intrinsic emission delay, counter response and readout uncertainty can all affect the observed probability density function.
Define the accepted lag interval by : is its full width. Integrating the flat baseline gives the existing accidental coincidence expectation . The additional area is . Moving a window of the same full width away from the peak tests the baseline under the model's stationarity assumption. It does not remove correlated instrumental backgrounds or changing rates automatically.
Lag-dependent pairs and a joint probability distribution carry information absent from singles totals
. Original chosen models, not historical counts or fitted data. Left: a normalized normal distribution of genuine-pair lags with standard deviation is added to an independent-pair baseline. Shaded intervals have full width , centered at zero and . Right: a separate exact ideal-bin model ignores timing spread; the two populations have identical singles expectations but different joint expectations. Details.For the lag model, choose , , , and timing standard deviation . A full width gives expected accidental coincidences and accepted genuine pairs, using the error function. The total is approximately , whereas the displaced interval from to has approximately expected pairs. A finite sample fluctuates around these expectations; these values are not measured counts or a significance calculation.
The right panel isolates the information lost by retaining singles totals alone. In a separate idealized model, stipulate independent bins, with Bernoulli distribution indicators for electron and photon detection in a bin. Set and . Independence predicts , hence one expected double detection. Alternatively chooseThese nonnegative probabilities sum to one and preserve both marginal probabilities, yet predict double detections in expectation. Their covariance is . This is an exact joint probability distribution illustrating pairing; it does not model the timing spread in the left panel or reconstruct either historical hypothesis quantitatively. Its lesson is why two singles totals cannot decide whether the observed events share a cause.
Posterior predictive probability 2026-10-05
A posterior predictive probability averages a future-event probability over the Bayesian posterior. For a Bernoulli distribution with unknown parameter , it is . Under Beta-binomial conjugacy with posterior , the predictive success probability is .
For nonnegative integer-valued independent random variables , factor the expected value of a product:The Cauchy product explains what this means for individual probability masses:Every way of splitting the total contributes once. This is convolution of independent random variables; multiplying probability generating functions performs the bookkeeping automatically.
For an original example, take independent and , using the Bernoulli distribution and binomial distribution. ThensoDirect enumeration of the six pairs gives the same probability masses. For example, total one collects and , with contributions . The coefficients sum to one. The expected value is ; since , the variance is , agreeing with .
Uncorrelated random variables are insufficient for the product identity. For a concrete counterexample, let be uniform on and set , . Then , but , whereas . Zero covariance controls one mixed moment, not the full joint probability mass function.
Proper scoring rule Created 2026-10-07 Updated 2026-10-14
A proper scoring rule makes reporting the true probability distribution optimal in expected value, for every true probability distribution in the allowed class of forecasts. In the reward convention, the inequality in the title holds for every report and true law . The expected value is always taken under , because the report does not change how the outcome is generated. In the loss convention the same definition readsThis requirement makes a proper scoring rule useful for eliciting probabilities and comparing predictive distributions: deliberately distorting a known forecast cannot improve its expected reward or expected loss. The reward convention and its strict version are set out in Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others (Section 1).
For the Bernoulli distribution with , the Brier score has expected loss , and the negative logarithmic scoring rule has expected loss . Both are minimized at the honest report . The linear probability score, by contrast, maximizes its expected reward at . These are statements about averaging over both possible outcomes with their actual probabilities.
Expected losses of the Brier score and logarithmic scoring rule favor ; the linear probability score rewards
. Details. The left panel displays each excess expected loss divided by that rule's expected loss at : for the Brier score and approximately for the logarithmic scoring rule. This normalization permits comparison of curve shapes and preserves the minimizing report, since it subtracts a constant and divides by a positive constant independent of . Zero means no excess expected loss. The right panel shows the unmodified linear probability score reward, with its maximizing direction labeled explicitly.
Randomly shifted lattice point process 2026-10-14
Let have the uniform distribution on and place a point at every , . This is a simple point process, locally finite, and a stationary point process: a translation only changes the phase modulo one, preserving its uniform distribution. For every Borel set , the Tonelli theorem givesIts consecutive points are exactly one unit apart, despite having the same intensity measure as a unit-rate Poisson point process. This is the one-dimensional translated-grid construction.[ref]
Spatial Point Processes and their Applications. Section 2.1, Example 2.3, printed pp. 26-27: randomly translated grids have constant intensity
Every half-open interval of length one contains exactly one point. For and , , each count has the Bernoulli distribution with parameter , andMore generally, an interval of length , with a nonnegative integer and , contains or points with probabilities and . Its count variance is , while a unit-rate Poisson point process has count variance .
Return period Created 2026-10-06 Updated 2026-10-14
A return period expresses how rare a threshold exceedance is in a specified time block. Let be the sample maximum in block , fix a threshold , and suppose is the same for every block. The conventional return period is blocks; blocks of duration give a time of . Thus an annual exceedance probability means a 100-year return period. This names a frequency scale, not a schedule or a guarantee of one exceedance in each century. The annual convention and the misleading “once every 100 years” interpretation are discussed in USGS: the 100-year flood.
To derive a waiting-time interpretation, assume the exceedance indicators are independent and identically distributed random variables with the Bernoulli distribution of parameter . Define , counting the successful block as well as all preceding blocks. The first Bernoulli trials must fail and the th must succeed, so independence gives the geometric distributionThe expected value can also be read directly from the geometric series: . This counts blocks from the starting boundary, not the exact time of an exceedance within its block. Counting only failed blocks instead would give expected value . The waiting law and its memorylessness of the geometric distribution are developed in Kyle Siegrist, Random Services: the geometric distribution.
For a horizon of complete years, at least one exceedance is the complement of no exceedances. Under the same independence and constant- assumptions,For the toy choice , this gives in 10 years, in 30 years, in 50 years, and in 100 years. Even after 100 years there is a chance of no exceedance. These numbers are exact model calculations rounded to one decimal place, not measured frequencies. The median waiting count is 69 years, since ; the expected value is 100 years. A mean and a median answer different questions.
The exceedance count has the binomial distribution under independence, with expected value . Thus has expected value one, although its probability of being at least one is only : some realizations contribute several exceedances. For , the memorylessness of the geometric distribution gives . A long quiet spell does not make the next independent year's exceedance overdue, and an exceedance this year does not force a quiet interval. Two specified consecutive years both exceed with probability in this toy model.
A reproducible tabletop experiment makes the assumptions tangible. Put 100 identical slips numbered 1 to 100 in a bag; after thoroughly mixing, draw one and record whether it is 100. Replace the slip and mix again before every draw. Treat each draw as one toy year, so the exceedance indicator is a Bernoulli random variable with , provided the sampling is uniform and successive draws are independent. Repeat complete 30-draw runs and record the fraction containing at least one 100; separately repeat runs stopped at the first 100 and record their waiting counts. The first fraction targets ; the waiting-count histogram targets . Replacement keeps fixed, mixing is intended to support independence, and repeated runs expose sampling variation. This is a proposed demonstration of the toy model; no experimental results are asserted here.
Stationarity supports a time-invariant annual probability distribution, but does not imply independence. For an extreme counterexample, draw just one Bernoulli random variable of parameter and set for every year. This is a stationary process, yet the probability of any exceedance in every finite horizon is ; from an arbitrary starting boundary the wait is infinite with probability . The reciprocal marginal frequency therefore need not equal that forward waiting expected value. With equal marginal probabilities and arbitrary dependence, only the general bounds follow, the upper bound being the union bound. Linearity of expectation still gives without independence. Mere zero correlation does not restore the product formula; even pairwise independence does not imply joint independence. If annual probabilities change to , independence instead gives , and one fixed return period no longer describes all years.
Sanov theorem Created 2026-09-24 Updated 2026-11-05
Let be independent and identically distributed on a finite alphabet with full-support probability mass function , and let be their empirical distribution. Sanov theorem gives a large deviation principle on the probability simplex, with rate functionusing natural logarithms. For any measurable set , with interior and closure relative to that probability simplex,The method of types explains the exponent: a type class has roughly words, each with probability ; the number of types grows only polynomially.[ref]
Information Theory and Statistics: A Tutorial by Imre Csiszár and Paul C. Shields. Section 2.1, Large deviations via types, printed pp. 429–434
For Bernoulli distribution observations with success probability , the event therefore has probability . The exponent is ; atypical empirical laws closest in relative entropy dominate the rare event.
Saturated statistical model Created 2026-10-05 Updated 2026-10-14
A saturated model has enough independently varying mean parameters to match every observed response or response cell. In individual Bernoulli distribution data its fitted probabilities are , and its maximum log-likelihood is zero under . It defines the reference in a binomial deviance.
Scoring rule Created 2026-10-07 Updated 2026-10-14
A scoring rule turns a probability distribution forecast and an observed outcome into a number. For example, a forecast of rain with probability should express uncertainty more usefully than a bare prediction of rain. A scoring rule lets us compare such forecasts after observations arrive, and its choice determines whether exaggerating confidence pays. A proper scoring rule rewards an honest account of uncertainty in expected value.
There are two sign conventions. A reward is maximized; a loss function is minimized. The conversion reverses every comparison. Here the linear probability score is a reward, while the binary Brier score and the negative logarithmic scoring rule are losses. Calling all three a "score" without specifying the direction would hide the central difference.
Throughout the binary examples, the random variable has a Bernoulli distribution with actual success probability , and the report assigns probability to and to . The value is stipulated by this statistical model; it is not inferred from data or a claim that a forecasting system is calibrated. In practice the true predictive distribution is unknown, and the relevant is the conditional probability of success given the information available when the forecast is issued. Propriety evaluates incentives for reporting that predictive distribution; it does not supply the unknown probabilities.
For independent Bernoulli distribution upper-triangular entries of a symmetric zero-diagonal adjacency matrix of a graph, put . For a unit vector , has sub-Gaussian variance proxy at most , by the Hoeffding lemma. Hence . The volumetric bound for Euclidean metric nets gives a -metric net of the unit sphere with at most points. The quadratic form net bound gives on that net. The union bound gives ; integrating this tail proves the displayed expectation bound.
Choose one of two regimes once, with equal probability: a source of independent and identically distributed random variables with one-bit probability , or one with one-bit probability . Keep the chosen regime forever. The resulting source is a strictly stationary process, because shifting a block does not change either component law. It is not an ergodic stationary process: its limiting empirical frequency identifies the regime.
The source's information rate isConditioning on the single regime bit changes any block's information entropy by at most one bit, so dividing by the block length proves this rate. However, the limiting realized information content per symbol is approximately in the first regime and one in the second. The alternative component's likelihood ratio decays exponentially along a typical path from the chosen component, at a rate given by the positive relative entropy between the two Bernoulli distributions. The mixture therefore has two possible pathwise limits, not one limit equal to the averaged information rate.
Statistical model Created 2026-09-24 Updated 2026-11-05
A statistical model is a specified family of probability distributions for the observations. A statistical parameter often indexes it as , where is the parameter space. The choice states which patterns of variation are considered possible and supplies a basis for statistical inference; it does not guarantee that any member exactly describes the data-generating mechanism.
For example, suppose binary observations are independent and identically distributed with a common unknown success probability . The Bernoulli distribution model has and likelihood functionWith and seven successes, this becomes and is maximized at . The binary observation type alone does not supply independence or a common : those are additional model assumptions. A regression model can instead let the response law depend on observed covariates, while a mixed model represents a specified grouping structure through random effects.
Strictly proper scoring rule Created 2026-10-07 Updated 2026-10-14
A strictly proper scoring rule is a proper scoring rule for which the true probability distribution is the unique optimizer of the expected value. Equivalently, any different admissible probability distribution gives strictly larger expected loss. For probability density functions, two densities that agree almost everywhere represent the same probability distribution. Ordinary propriety permits ties: the constant reward is a proper scoring rule, because every report ties, but it is not a strictly proper scoring rule and elicits no information.
Strict propriety concerns expectation, not guaranteed success on each observation or each realized dataset. With the stipulated Bernoulli distribution , suppose the outcome is . The honest report then has Brier score loss , while has loss . The logarithmic loss similarly gives losses and . The honest forecast loses this comparison on that draw, which occurs with probability in the stipulated statistical model.
Individual outcomes can favor a different report even though the expected Brier score and logarithmic loss are uniquely minimized at
. Details. The loss curves favor , and the loss curves favor . Their weighted average, with weights and , favors . Choosing a report after observing would use information unavailable at forecast time. Even an average loss on a finite dataset can rank an honest forecast below a competing forecast through outcome variation; strict propriety guarantees the ranking in expected value under the specified information and true predictive distribution.




