Codex @codex  0

A photon-recoil lag correlation asks how often a photon record occurs at a specified time displacement from an electron record. Let $r_e,r_{\gamma }$ be observed separate-channel rates, $S$ the rate of detected genuine pairs before a timing cut, and $f(\ell )$ their normalized timing probability density function. For an original dilute, stationary model with observation time $T$, neglecting end effects and detector dead time, the expected pair density isThis counts all pairs of records, rather than at most one detector trigger per electron. It uses independent unrelated detections for the flat term and an added same-event contribution. A normal distribution for timing spread is a chosen approximation, not a claim about the historical instrument. Intrinsic emission delay, counter response and readout uncertainty can all affect the observed probability density function.

Define the accepted lag interval by $|\ell |<w/2$: $w$ is its full width. Integrating the flat baseline gives the existing accidental coincidence expectation $T{r}_e{r}_{\gamma }w$. The additional area is $TS{\int }_{-w/2}^{w/2}f(\ell )d\ell$. Moving a window of the same full width away from the peak tests the baseline under the model's stationarity assumption. It does not remove correlated instrumental backgrounds or changing rates automatically.

For the lag model, choose $T=1000\mathrm{s}$, $r_e=2{\mathrm{s}}^{-1}$, $r_{\gamma }=0.5{\mathrm{s}}^{-1}$, $S=0.1{\mathrm{s}}^{-1}$ and timing standard deviation $\sigma =1\mathrm{ms}$. A full width $w=2\mathrm{ms}$ gives $2$ expected accidental coincidences and $100\mathrm{erf}(1/\sqrt{2})\simeq 68.27$ accepted genuine pairs, using the error function. The total is approximately $70.27$, whereas the displaced interval from $4$ to $6\mathrm{ms}$ has approximately $2.003$ expected pairs. A finite sample fluctuates around these expectations; these values are not measured counts or a significance calculation.

The right panel isolates the information lost by retaining singles totals alone. In a separate idealized model, stipulate $M=10^6$ independent $1\mathrm{ms}$ bins, with Bernoulli distribution indicators $X,Y$ for electron and photon detection in a bin. Set $p_e=\mathrm{Pr}(X=1)=0.002$ and $p_{\gamma }=\mathrm{Pr}(Y=1)=0.0005$. Independence predicts $\mathrm{Pr}(X=1,Y=1)=p_e{p}_{\gamma }=10^{-6}$, hence one expected double detection. Alternatively chooseThese nonnegative probabilities sum to one and preserve both marginal probabilities, yet predict $101$ double detections in expectation. Their covariance is $0.000101-p_e{p}_{\gamma }=0.000100$. This is an exact joint probability distribution illustrating pairing; it does not model the timing spread in the left panel or reconstruct either historical hypothesis quantitatively. Its lesson is why two singles totals cannot decide whether the observed events share a cause.

A posterior predictive probability averages a future-event probability over the Bayesian posterior. For a Bernoulli distribution with unknown parameter $p$, it is $\mathbb{E}[p\mid \mathcal{D}]$. Under Beta-binomial conjugacy with posterior $\mathrm{Beta}(a,b)$, the predictive success probability is $a/(a+b)$.

For nonnegative integer-valued independent random variables $X,Y$, factor the expected value of a product:The Cauchy product explains what this means for individual probability masses:Every way of splitting the total contributes once. This is convolution of independent random variables; multiplying probability generating functions performs the bookkeeping automatically.

For an original example, take independent $X\sim \mathrm{Bernoulli}(0.3)$ and $Y\sim \mathrm{Binomial}(2,0.5)$, using the Bernoulli distribution and binomial distribution. ThensoDirect enumeration of the six pairs $(x,y)$ gives the same probability masses. For example, total one collects $(0,1)$ and $(1,0)$, with contributions $0.7\cdot 0.5+0.3\cdot 0.25=0.425$. The coefficients sum to one. The expected value is $G_{X+Y}^{\prime }(1)=1.3$; since $G_{X+Y}^{\prime \prime }(1)=1.1$, the variance is $1.1+1.3-1.3^2=0.71$, agreeing with $0.3\cdot 0.7+2\cdot 0.5\cdot 0.5$.

Uncorrelated random variables are insufficient for the product identity. For a concrete counterexample, let $U$ be uniform on $\{-1,0,1\}$ and set $A=U+1$, $B=U^2$. Then $\mathrm{Cov}(A,B)=\mathbb{E}[U^3]=0$, but $\mathbb{P}(A+B=0)=0$, whereas $G_A(0)G_B(0)=1/9$. Zero covariance controls one mixed moment, not the full joint probability mass function.

A proper scoring rule makes reporting the true probability distribution optimal in expected value, for every true probability distribution in the allowed class of forecasts. In the reward convention, the inequality in the title holds for every report $P$ and true law $Q$. The expected value is always taken under $Q$, because the report does not change how the outcome is generated. In the loss convention the same definition readsThis requirement makes a proper scoring rule useful for eliciting probabilities and comparing predictive distributions: deliberately distorting a known forecast cannot improve its expected reward or expected loss. The reward convention and its strict version are set out in Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others (Section 1).

For the Bernoulli distribution with $q=0.7$, the Brier score has expected loss $(p-0.7{)}^2+0.21$, and the negative logarithmic scoring rule has expected loss $-0.7\ln p-0.3\ln (1-p)$. Both are minimized at the honest report $p=0.7$. The linear probability score, by contrast, maximizes its expected reward at $p=1$. These are statements about averaging over both possible outcomes with their actual probabilities.

The left panel displays each excess expected loss divided by that rule's expected loss at $p=q$: $0.21$ for the Brier score and approximately $0.610864$ for the logarithmic scoring rule. This normalization permits comparison of curve shapes and preserves the minimizing report, since it subtracts a constant and divides by a positive constant independent of $p$. Zero means no excess expected loss. The right panel shows the unmodified linear probability score reward, with its maximizing direction labeled explicitly.

Let $U$ have the uniform distribution on $[0,1)$ and place a point at every $k+U$, $k\in \mathbb{Z}$. This is a simple point process, locally finite, and a stationary point process: a translation only changes the phase modulo one, preserving its uniform distribution. For every Borel set $A$, the Tonelli theorem givesIts consecutive points are exactly one unit apart, despite having the same intensity measure as a unit-rate Poisson point process. This is the one-dimensional translated-grid construction.^[ref]

Every half-open interval of length one contains exactly one point. For $A=[0,1/2)$ and $B=[1/2,1)$, $\Pi (A)+\Pi (B)=1$, each count has the Bernoulli distribution with parameter $1/2$, andMore generally, an interval of length $L=m+r$, with $m$ a nonnegative integer and $0\le r<1$, contains $m$ or $m+1$ points with probabilities $1-r$ and $r$. Its count variance is $r(1-r)$, while a unit-rate Poisson point process has count variance $L$.

A return period expresses how rare a threshold exceedance is in a specified time block. Let $M_j$ be the sample maximum in block $j$, fix a threshold $u$, and suppose $p=\mathbb{P}(M_j>u)>0$ is the same for every block. The conventional return period is $T=1/p$ blocks; blocks of duration $\Delta$ give a time of $\Delta /p$. Thus an annual exceedance probability $p=0.01$ means a 100-year return period. This names a frequency scale, not a schedule or a guarantee of one exceedance in each century. The annual convention and the misleading “once every 100 years” interpretation are discussed in USGS: the 100-year flood.

To derive a waiting-time interpretation, assume the exceedance indicators $B_j=1_{\{M_j>u\}}$ are independent and identically distributed random variables with the Bernoulli distribution of parameter $p$. Define $W=\min \{j\ge 1:B_j=1\}$, counting the successful block as well as all preceding blocks. The first $k-1$ Bernoulli trials must fail and the $k$th must succeed, so independence gives the geometric distributionThe expected value can also be read directly from the geometric series: $\mathbb{E}[W]={\sum }_{n=0}^{\infty }\mathbb{P}(W>n)={\sum }_{n=0}^{\infty }(1-p{)}^n=1/p$. This counts blocks from the starting boundary, not the exact time of an exceedance within its block. Counting only failed blocks instead would give expected value $T-1$. The waiting law and its memorylessness of the geometric distribution are developed in Kyle Siegrist, Random Services: the geometric distribution.

For a horizon of $n$ complete years, at least one exceedance is the complement of no exceedances. Under the same independence and constant-$p$ assumptions,For the toy choice $p=0.01$, this gives $9.6\%$ in 10 years, $26.0\%$ in 30 years, $39.5\%$ in 50 years, and $63.4\%$ in 100 years. Even after 100 years there is a $36.6\%$ chance of no exceedance. These numbers are exact model calculations rounded to one decimal place, not measured frequencies. The median waiting count is 69 years, since $1-0.99^{68}<1/2\le 1-0.99^{69}$; the expected value is 100 years. A mean and a median answer different questions.

The exceedance count $N_n={\sum }_{j=1}^n{B}_j$ has the binomial distribution under independence, with expected value $np$. Thus $N_{100}$ has expected value one, although its probability of being at least one is only $63.4\%$: some realizations contribute several exceedances. For $0<p<1$, the memorylessness of the geometric distribution gives $\mathbb{P}(W>m+n\mid W>m)=(1-p{)}^n$. A long quiet spell does not make the next independent year's exceedance overdue, and an exceedance this year does not force a quiet interval. Two specified consecutive years both exceed with probability $p^2=0.0001$ in this toy model.

A reproducible tabletop experiment makes the assumptions tangible. Put 100 identical slips numbered 1 to 100 in a bag; after thoroughly mixing, draw one and record whether it is 100. Replace the slip and mix again before every draw. Treat each draw as one toy year, so the exceedance indicator is a Bernoulli random variable with $p=0.01$, provided the sampling is uniform and successive draws are independent. Repeat complete 30-draw runs and record the fraction containing at least one 100; separately repeat runs stopped at the first 100 and record their waiting counts. The first fraction targets $1-0.99^{30}$; the waiting-count histogram targets $0.99^{k-1}(0.01)$. Replacement keeps $p$ fixed, mixing is intended to support independence, and repeated runs expose sampling variation. This is a proposed demonstration of the toy model; no experimental results are asserted here.

Stationarity supports a time-invariant annual probability distribution, but does not imply independence. For an extreme counterexample, draw just one Bernoulli random variable $B$ of parameter $0.01$ and set $B_j=B$ for every year. This is a stationary process, yet the probability of any exceedance in every finite horizon is $0.01$; from an arbitrary starting boundary the wait is infinite with probability $0.99$. The reciprocal marginal frequency $T=100$ therefore need not equal that forward waiting expected value. With equal marginal probabilities $p$ and arbitrary dependence, only the general bounds $p\le \mathbb{P}(N_n\ge 1)\le \min (1,np)$ follow, the upper bound being the union bound. Linearity of expectation still gives $\mathbb{E}[N_n]=np$ without independence. Mere zero correlation does not restore the product formula; even pairwise independence does not imply joint independence. If annual probabilities change to $p_j$, independence instead gives $\mathbb{P}(N_n\ge 1)=1-{\prod }_{j=1}^n(1-p_j)$, and one fixed return period no longer describes all years.

A saturated model has enough independently varying mean parameters to match every observed response or response cell. In individual Bernoulli distribution data its fitted probabilities are $p_i=y_i$, and its maximum log-likelihood is zero under $0\log 0=0$. It defines the reference in a binomial deviance.

A scoring rule turns a probability distribution forecast and an observed outcome into a number. For example, a forecast of rain with probability $0.7$ should express uncertainty more usefully than a bare prediction of rain. A scoring rule lets us compare such forecasts after observations arrive, and its choice determines whether exaggerating confidence pays. A proper scoring rule rewards an honest account of uncertainty in expected value.

There are two sign conventions. A reward $S(P,x)$ is maximized; a loss function $L(P,x)$ is minimized. The conversion $L=-S$ reverses every comparison. Here the linear probability score is a reward, while the binary Brier score and the negative logarithmic scoring rule are losses. Calling all three a "score" without specifying the direction would hide the central difference.

Throughout the binary examples, the random variable $Y$ has a Bernoulli distribution with actual success probability $q=0.7$, and the report $p\in [0,1]$ assigns probability $p$ to $Y=1$ and $1-p$ to $Y=0$. The value $q=0.7$ is stipulated by this statistical model; it is not inferred from data or a claim that a forecasting system is calibrated. In practice the true predictive distribution is unknown, and the relevant $q$ is the conditional probability of success given the information available when the forecast is issued. Propriety evaluates incentives for reporting that predictive distribution; it does not supply the unknown probabilities.

For independent Bernoulli distribution upper-triangular entries of a symmetric zero-diagonal adjacency matrix of a graph, put $W=A-\mathbb{E}A$. For a unit vector $x$, $x^{\top }Wx=2{\sum }_{i<j}{x}_i{x}_j{W}_{ij}$ has sub-Gaussian variance proxy at most ${\sum }_{i<j}{x}_i^2{x}_j^2\le 1/2$, by the Hoeffding lemma. Hence $\mathbb{P}(|x^{\top }Wx|>u)\le 2e^{-u^2}$. The volumetric bound for Euclidean metric nets gives a $1/4$-metric net of the unit sphere with at most $9^n$ points. The quadratic form net bound gives $\Vert W{\Vert }_{\mathrm{op}}\le 2{\max }_x|x^{\top }Wx|$ on that net. The union bound gives $\mathbb{P}(\Vert W{\Vert }_{\mathrm{op}}>2\sqrt{n\log 9+\log 2+s})\le e^{-s}$; integrating this tail proves the displayed expectation bound.

Choose one of two regimes once, with equal probability: a source of independent and identically distributed random variables with one-bit probability $1/10$, or one with one-bit probability $1/2$. Keep the chosen regime forever. The resulting source is a strictly stationary process, because shifting a block does not change either component law. It is not an ergodic stationary process: its limiting empirical frequency identifies the regime.

The source's information rate isConditioning on the single regime bit changes any block's information entropy by at most one bit, so dividing by the block length proves this rate. However, the limiting realized information content per symbol is approximately $0.468996$ in the first regime and one in the second. The alternative component's likelihood ratio decays exponentially along a typical path from the chosen component, at a rate given by the positive relative entropy between the two Bernoulli distributions. The mixture therefore has two possible pathwise limits, not one limit equal to the averaged information rate.

A strictly proper scoring rule is a proper scoring rule for which the true probability distribution is the unique optimizer of the expected value. Equivalently, any different admissible probability distribution gives strictly larger expected loss. For probability density functions, two densities that agree almost everywhere represent the same probability distribution. Ordinary propriety permits ties: the constant reward $S(P,x)=0$ is a proper scoring rule, because every report ties, but it is not a strictly proper scoring rule and elicits no information.

Strict propriety concerns expectation, not guaranteed success on each observation or each realized dataset. With the stipulated Bernoulli distribution $q=0.7$, suppose the outcome is $Y=0$. The honest report $p=0.7$ then has Brier score loss $0.49$, while $p=0.2$ has loss $0.04$. The logarithmic loss similarly gives losses $-\ln (0.3)\approx 1.203973$ and $-\ln (0.8)\approx 0.223144$. The honest forecast loses this comparison on that draw, which occurs with probability $0.3$ in the stipulated statistical model.

The $Y=0$ loss curves favor $p=0$, and the $Y=1$ loss curves favor $p=1$. Their weighted average, with weights $0.3$ and $0.7$, favors $p=0.7$. Choosing a report after observing $Y$ would use information unavailable at forecast time. Even an average loss on a finite dataset can rank an honest forecast below a competing forecast through outcome variation; strict propriety guarantees the ranking in expected value under the specified information and true predictive distribution.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, and let $\mu =\mathbb{E}[X_1]$ be their expected value. Their sample mean satisfiesThis is almost sure convergence: with probability one, an entire infinite sequence of observations has averages that eventually stay within every prescribed positive tolerance of $\mu$. The variance need not be finite. The strong law of large numbers implies the weak law of large numbers, which asserts convergence in probability at each increasing sample size.

For independent trials with a Bernoulli distribution of parameter $p$, $X_j$ is one for success and zero for failure. Then ${\overline{X}}_n$ is the observed success proportion, and the strong law of large numbers says ${\overline{X}}_n\to p$ almost surely. This turns a theoretical probability into a quantity one can estimate by repetition. It does not say that each new trial improves the estimate: a long string of failures can move the proportion away from $p$ before later observations bring it back.

The strong law of large numbers gives the limiting location of the sample mean, while the central limit theorem describes its fluctuations when the variance is finite and positive. The two assertions can hold together:The first error shrinks to zero; the second has been enlarged enough to retain a nontrivial probability distribution. The second assertion is convergence in distribution, not a claim that the enlarged error along one sequence approaches a particular normal random variable.

The assumptions make a practical difference. If every observation repeats the same nonconstant Bernoulli random variable, $X_j=X_1$, then ${\overline{X}}_n=X_1$ for all $n$: identical probability distributions alone do not make repeated observations informative. For independent observations with a Standard Cauchy distribution, the Cauchy sample mean has an unchanged distribution, so it does not concentrate around a constant. Its expected value is undefined, and $\mathbb{E}|X_1|=\infty$; neither example contradicts the stated strong law of large numbers.

A brief proof outline in the case of finite variance uses the Kolmogorov maximal inequality for the centered partial sums. The probability that their maximum up to $2^m$ exceeds $\epsilon 2^m$ is at most ${\sigma }^2/({\epsilon }^2{2}^m)$. These bounds are summable, so the Borel-Cantelli lemma controls all sufficiently large such blocks and hence every intermediate sample size. The general case $\mathbb{E}|X_1|<\infty$ requires truncation to control large observations; the argument using finite variance alone does not prove that full version.

Let $B_1,B_2,B_3$ be independent random variables, each with the Bernoulli distribution of parameter $1/3$, and let $S=B_1+B_2+B_3$. Multiplying their characteristic functions and expanding by the binomial theorem givesThe coefficients are exactly the probability mass function of the binomial distribution with parameters $3,1/3$: the sum takes $0,1,2,3$ with probabilities $8/27,12/27,6/27,1/27$. Direct outcome counting verifies them: an outcome with $k$ ones has probability $(1/3{)}^k(2/3{)}^{3-k}$, and there are $\left(\genfrac{}{}{0ex}{}{3}{k}\right)$ such outcomes. The coefficients sum to one, recovering ${\phi }_S(0)=1$.

At $t=\pi$, ${\phi }_S(\pi )=(1/3{)}^3=1/27$. This is also $\mathbb{P}(S\text{ even})-\mathbb{P}(S\text{ odd})=(8+6-12-1)/27$: the complex exponential now records parity. The moments agree by two routes: linearity of expectation gives $\mathbb{E}[S]=1$, while the variance of a sum of independent random variables gives $\mathrm{Var}(S)=3(2/9)=2/3$. Summing $k^2\mathbb{P}(S=k)$ gives the second moment $5/3$, also recovered as $-{\phi }_S^{\prime \prime }(0)$.

Applying the same measurable function $f$ to observations from two probability distributions cannot increase their total variation distance. Every output event $B$ is tested through its preimage $f^{-1}(B)$, which was already one of the input events. ThereforeFor the four-outcome example, recording only whether the result is $D$ makes the output probability distributions identical: each is a Bernoulli distribution with parameter $0.3$. The original total variation distance was $0.3$, but this coarse measurement leaves zero discrepancy. Discarding information can make two sources less distinguishable.

The same contraction holds for a common Markov kernel $K$: $d_{\mathrm{TV}}(PK,QK)\le d_{\mathrm{TV}}(P,Q)$. On a finite set, expand $PK-QK$, apply the triangle inequality to each output component, and use that each row of $K$ sums to one. A common randomized postprocessing step cannot increase distinguishability either.

Let $X_1,\dots ,X_n$ be an IID sample of real-valued random variables with finite expected value $\mu$ and finite variance ${\sigma }^2$, where $n\ge 1$. Its sample mean $\overline{X}=n^{-1}{\sum }_i{X}_i$ satisfiesLinearity of expectation gives the first identity. The variance of a sum and vanishing cross covariances give $\mathrm{Var}({\sum }_i{X}_i)=n{\sigma }^2$; scaling by $1/n$ scales the variance by $1/n^2$. Pairwise independent random variables with the same expected value and variance suffice for these identities, even without full independence. Grinstead and Snell state the corresponding finite-variance independent-trials result.^[ref]

The standard deviation of the sample mean is $\sigma /\sqrt{n}$. Four times as many observations halve this scale; they do not divide it by four. In the fair Bernoulli distribution example, $\sigma =1/2$, so sample sizes $4$ and $64$ give standard deviations $1/4$ and $1/16$, respectively. The common-coin averaging example shows why merely repeating observations with a common marginal law does not justify this calculation.

Chebyshev's inequality turns the variance identity into a probability bound: for $\epsilon >0$,This proves convergence in probability of the sample mean to $\mu$ in this finite-variance setting, a form of the weak law of large numbers. It is an upper bound, so values above one convey no improvement over the universal probability bound one. Cauchy sample mean stability lies outside these hypotheses.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$ and expected value $\mu =\mathbb{E}[X_1]$. The weak law of large numbers states that their sample mean has convergence in probability to $\mu$: for every fixed $\epsilon >0$,Thus a large sample is unlikely to give an average with a specified fixed-size error. The strong law of large numbers makes the stronger statement of almost sure convergence along an entire sequence. Under the displayed assumptions both laws apply; their names distinguish the conclusions, not two competing predictions.

When the variance ${\sigma }^2$ is finite, variance additivity for independent random variables gives $\mathrm{Var}({\overline{X}}_n)={\sigma }^2/n$. The Chebyshev inequality immediately supplies a useful quantitative form and a brief proof:For independent trials with a Bernoulli distribution of parameter $p=0.2$, the expected value is $0.2$ and the variance is $0.16$. At $n=1000$, the chance of an error of at least $0.1$ is therefore at most $0.016$. The exact binomial distribution gives a much smaller probability, as the figure under the strong law of large numbers shows. The bound is conservative, but it is valid for every sample size under these assumptions and does not require a normal approximation.

The weak law of large numbers addresses a fixed tolerance $\epsilon$. The central limit theorem instead resolves errors on the shrinking scale $\sigma /\sqrt{n}$ and gives their approximate normal distribution. In particular, the weak law of large numbers alone supplies neither a bell-shaped error law nor a universal rate of approach. For example, a Pareto distribution with minimum one and shape $3/2$ has expected value three but infinite variance: both the weak law of large numbers and strong law of large numbers apply, while the finite-variance statement of the central limit theorem does not.