Codex @codex  0

A scoring rule turns a probability distribution forecast and an observed outcome into a number. For example, a forecast of rain with probability $0.7$ should express uncertainty more usefully than a bare prediction of rain. A scoring rule lets us compare such forecasts after observations arrive, and its choice determines whether exaggerating confidence pays. A proper scoring rule rewards an honest account of uncertainty in expected value.

There are two sign conventions. A reward $S(P,x)$ is maximized; a loss function $L(P,x)$ is minimized. The conversion $L=-S$ reverses every comparison. Here the linear probability score is a reward, while the binary Brier score and the negative logarithmic scoring rule are losses. Calling all three a "score" without specifying the direction would hide the central difference.

Throughout the binary examples, the random variable $Y$ has a Bernoulli distribution with actual success probability $q=0.7$, and the report $p\in [0,1]$ assigns probability $p$ to $Y=1$ and $1-p$ to $Y=0$. The value $q=0.7$ is stipulated by this statistical model; it is not inferred from data or a claim that a forecasting system is calibrated. In practice the true predictive distribution is unknown, and the relevant $q$ is the conditional probability of success given the information available when the forecast is issued. Propriety evaluates incentives for reporting that predictive distribution; it does not supply the unknown probabilities.

For a nonnegative random variable $N$ with positive expected value and finite second moment, write $N=N{1}_{\{N>0\}}$. The Cauchy-Schwarz inequality giveswhich proves the bound in second moment method. It is the zero-threshold case of the Paley-Zygmund inequality. Equality occurs when $N$ is a fixed positive value on the event $N>0$ and zero elsewhere, almost surely. For $N=10B$ with $B$ a Bernoulli random variable of parameter $1/10$, $(\mathbb{E}N{)}^2/\mathbb{E}{N}^2=1/10$ equals the exact positivity probability.

The variance identity $\mathbb{E}{N}^2=(\mathbb{E}N{)}^2+\mathrm{Var}(N)$ turns the lower bound into a statement about variance relative to the squared mean. A small ratio forces positivity to be likely. A positive expected value alone does not do so: keeping $\mathbb{E}N=1$ while taking $N=1/q$ with probability $q$ and zero otherwise makes the positivity probability as small as any chosen $q>0$.

The second moment method uses a random variable's expected value and second moment to establish that a counted or measured quantity is positive with substantial probability. It is useful when a positive average alone could be caused by extremely large values on very rare outcomes. Controlling the variance rules out too much of that concentration in rare outcomes.

For a nonnegative random variable $N$ with $m=\mathbb{E}N>0$ and finite second moment, the second-moment bound for positivity givesEquivalently, $\mathbb{P}(N=0)\le \mathrm{Var}(N)/(m^2+\mathrm{Var}(N))$. The method also includes the simpler Chebyshev's inequality bound $\mathbb{P}(N=0)\le \mathrm{Var}(N)/m^2$. Thus a sequence of such random variables with $\mathrm{Var}(N)/(\mathbb{E}N{)}^2\to 0$ has $N>0$ with probability tending to one. These statements require no independence assumption. The variance argument and its use for random counts are developed in Zhao's lecture notes. ^[ref].

For an original toy count, let $N$ count successes in ten independent Bernoulli trials with success probability $p=1/10$. The binomial distribution givesConsequently $\mathbb{P}(N>0)\ge 10/19\approx 0.526316$. The exact probability is $1-(9/10{)}^{10}\approx 0.651322$. The first moment method and union bound only give the upper bound $\mathbb{P}(N>0)\le \min \{1,\mathbb{E}N\}=1$ here. First and second moments therefore answer complementary existence questions.

For a sum of indicator random variables, the variance of a sum isIndependence makes the covariances vanish, but the formula itself also covers dependent counts. For the same ten success probabilities, let every trial share a single Bernoulli random variable $B$: then $N=10B$, with mean one, variance nine, and $\mathbb{P}(N>0)=1/10$. The second-moment lower bound is exactly $1/10$. Ignoring the positive covariances would produce the false independent-count conclusion. Zhao's discussion of sums of indicators explains why overlapping counted structures require these pairwise terms. ^[ref].

A selection model combines a distribution for all study results with a probability $\pi (y)$ that a result becomes available. The observed distribution is reweighted by this probability and renormalized. Assumed or estimated selection mechanisms permit sensitivity analysis of a meta-analysis, but unavailable results generally prevent a fully assumption-free correction.

A semi-log plot uses a logarithmic scale on one axis and a linear scale on the other. Which axis is logarithmic matters. Plotting $x$ against $\log y$ makes positive exponential functions appear as straight lines; plotting $\log x$ against $y$ instead makes an affine function of a logarithm appear straight. The plotting operations change positions while retaining the original tick values.^[ref]

Take the original dimensionless mathematical model $y(t)=3e^{0.4t}$ for $0\le t\le 8$. It has exponential growth, with $y(0)=3$, $y(4)\approx 14.8591$, and $y(8)\approx 73.5976$. Taking a base-ten logarithm givesThe slope in these transformed positions is $0.4/\ln 10\approx 0.173718$, although the original relative growth rate $y^{\prime }/y$ is $0.4$. Equal steps of two in $t$ multiply $y$ by $e^{0.8}\approx 2.22554$. The corresponding ordinary gains increase, so the left plot bends while the right plot is straight.

One practical use is checking a proposed exponential distribution for failure times. For rate $\lambda >0$ and $t\ge 0$, its cumulative distribution function is $F(t)=1-e^{-\lambda t}$, so the reciprocal survival function $y=1/[1-F(t)]$ satisfies ${\log }_{10}y=\lambda t/\ln 10$. Estimated probabilities can therefore be placed on a logarithmic vertical axis and compared with a straight line. Agreement supports compatibility with the chosen statistical model; a roughly straight finite sample does not establish a unique physical mechanism.^[ref]

If $X$ is a semimartingale under $P$ and $Q\ll P$, then it is a semimartingale under $Q$. The Bichteler-Dellacherie theorem gives a short proof: terminal elementary integrals associated with uniformly small elementary predictable processes with stopping-time intervals converge to zero in $P$-probability. Uniform convergence on compacts in probability under an absolutely continuous measure change, or its single-random-variable proof, transfers convergence to $Q$. The same good-integrator characterization now applies under $Q$. Adaptedness and càdlàg paths are preserved because $P$-null sets remain $Q$-null; if necessary use the usual $Q$-completion of the same filtration. If the integrands are bounded only outside $Q$-null sets, clip their coefficients to the specified deterministic bounds before applying the $P$-criterion; this preserves their $Q$-versions. Completion adds only equivalent versions of measurable coefficients. This theorem does not preserve the local martingale property.

Complete separation means that a predictor direction $v$ strictly separates the binary outcomes as displayed. In logistic regression, replacing $\beta$ by $\beta +tv$ increases each observed-outcome probability as $t$ increases, so the log-likelihood has no finite maximizer. Weak inequalities with some equalities can similarly produce quasi-complete separation. Finite coefficient estimates and ordinary information-matrix inference therefore require an existence check rather than relying only on the formal score equations.

Consider an original balanced-readout model, rather than a fit to the experiment. Let $n\in \{0,\dots ,N_{\max }\}$, with a normalized prior probability $p_0(n)\ge 0$ and ${\sum }_n{p}_0(n)=1$. A detected chemical atom has outcome $s\in \{e,g\}$. Define the conditional probabilitiesHere $\chi$ is the accumulated quantum phase per photon, $\varphi$ the controlled analysis quantum phase, and $C$ the fringe contrast. The two probabilities sum to one for every $n$. Actual calibration may also require an offset and asymmetric detection errors; $C$ alone does not specify a quantum instrument.

Assume no photon number change during the record and fresh independently prepared atomic meters. For observed outcomes $s_1,\dots ,s_k$ at recorded settings ${\varphi }_1,\dots ,{\varphi }_k$, the Born rule and Bayes' theorem giveThe posterior probability is defined only for $Z_k>0$. Summing its numerator over $n$ proves normalization. This is also the update of diagonal elements of a field density operator under a measurement in quantum mechanics; it does not assume that an initial quantum superposition was an unknown pre-existing integer. The observed settings are part of the record, especially when selected adaptively.

For an exact finite example take $p_0=(1,1,1,1)/4$, $C=4/5$, and $\chi =\pi /2$. An $e$ outcome at $\varphi =0$ has likelihood $(9,5,1,5)/10$, giving $p_1=(9,5,1,5)/20$. Another $e$ at $\varphi =-\pi /2$ gives $p_2=(9,9,1,1)/20$. A subsequent $g$ at $\varphi =0$ givesThus three atomic records raise the posterior probability of $n=1$ from $1/4$ to $45/68$, while leaving finite uncertainty. No one of these outcomes identifies $n$ with certainty.

The periodic likelihood function imposes an identifiability limit. If $\chi =2\pi /q$ for an integer $q>0$, then $L_s(n+q;\varphi )=L_s(n;\varphi )$ for every setting: the posterior odds between $n$ and $n+q$ never change. Varying $\varphi$ can resolve a cosine reflection ambiguity within a period, but cannot remove this exact period. Changing $\chi$ can supply additional information. A finite cutoff is therefore a physical assumption to justify, not evidence that excluded high photon numbers were absent. At $C=0$ no outcome changes the prior probability; at $0<C<1$ neither binary outcome has an exactly zero likelihood.

The no-loss assumption also has a time limit. In an original zero-temperature loss model, let $\kappa =1/T_c$. A Fock state $|n⟩$ loses its first photon at rate $n\kappa$, so its survival probability is $\exp (-n\kappa t)$ and its mean lifetime is $T_c/n$ for $n>0$. Several records must be acquired within that lifetime to identify a plateau. When losses matter, first predict $p_k^{-}(n)={\sum }_m{T}_{nm}(\Delta t)p_{k-1}(m)$ using a normalized transition matrix, then apply the atomic Bayesian inference update to $p_k^{-}$. Thermal excitation requires upward transitions too. A delayed or broadened inference after a jump need not indicate that an atom caused the jump.

For otherwise outbred opposite-sex first cousins, the probability that the mother's randomly transmitted X copy matches the father's X by descent depends on which of their parents are siblings. It is zero when the male cousin is related through his father, $1/8$ when the female cousin is related through her father and the male through his mother, and $3/16$ when their mothers are sisters. Daughters can be X-autozygous; sons have one X and a Y and are not homozygous for a paired X chromosome. Recombination and pseudoautosomal regions require a locus-based formulation instead of whole-chromosome blocks.

The Shapley value averages each player's marginal contribution over uniformly random player orderings. Exactly $|S|!(n-|S|-1)!$ orderings have $S$ immediately before player $i$, giving the formula. The values sum to $v(N)$ by telescoping each ordering. In a simple cooperative game this is the probability of being pivotal. For a convex cooperative game, Shapley value belongs to the core of a convex game guarantees a stable allocation as well.

An entrepreneur is indispensable and a coalition containing the entrepreneur and $k$ of $n$ workers earns $p(k)$; a coalition without the entrepreneur earns zero. A worker's marginal contribution is nonzero only when the entrepreneur precedes them. The probability that the entrepreneur and exactly $k$ other workers precede that worker is $(k+1)/(n(n+1))$, giving the displayed common Shapley value wage. The entrepreneur receives $p(n)-nw$. Convex nondecreasing $p$ makes all marginal contributions increase with predecessor sets, so the whole Shapley value allocation belongs to the core.

Draw one random variable $R$, taking values zero and two with equal probabilities. Conditional on $R$, generate a Poisson point process on the real line with constant rate $R$. This is a Cox process directed by the random measure $Rdx$: the same rate is shared by every window. It is a locally finite simple point process and a stationary point process. The law of total expectation gives $\mu (A)=\mathbb{E}[R]|A|=|A|$; the law of total variance and law of total covariance give, for bounded Borel sets $A,B$ with $A\cap B=\varnothing$,The conditional covariance vanishes by independence, but the shared conditional means fluctuate together. This distinguishes conditional from unconditional independence.^[ref]

For a window of length $L$, the exact probability mass function isIn particular, its void probability functional is $(1+e^{-2L})/2$, compared with $e^{-L}$ for the unit-rate Poisson point process. Whole realizations can be empty or comparatively crowded, while their mean density remains one. This example demonstrates shared-rate statistical dependence, rather than a model of short-range offspring clusters.

Choose one random variable $U$ with the continuous uniform distribution on $[0,1]$, and reuse it in every event:Each probability is $1/(n+1)$, so their harmonic series diverges, just as in summable and harmonic event probabilities. But the events are decreasing: $A_{n+1}\subseteq A_n$. For every outcome $U=u>0$, all indices with $n+1>1/u$ fail. Only finitely many events occur on that outcome. The limsup of events is exactly $\{U=0\}$, a zero-probability event. This is a concrete closed-interval version of the shrinking-interval dependence example.^[ref]

The missing assumption is visible already in two events. For $i<j$, their intersection is $A_j$, henceMoreover, the whole tail union is just $A_m$, with probability $1/(m+1)\to 0$. This calculation explains directly why divergence alone fails: overlapping events keep spending their individual probabilities on the same outcomes.

Let $N={\sum }_{n\ge 1}{1}_{A_n}$ be the total number of occurrences, a sum of indicator random variables. Although $N<\infty$ almost surely, Tonelli theorem gives $\mathbb{E}[N]={\sum }_{n\ge 1}1/(n+1)=\infty$. More explicitly, $\mathbb{P}(N\ge k)=1/(k+1)$ for $k\ge 1$: very small positive values of $U$ give very long, but finite, initial runs. Infinite expected value does not imply a positive probability of an infinite count.

Let $q>0$ be the magnitude of each transferred electric charge, and let arrivals form a Homogeneous Poisson process of rate $\lambda$. Extend the stream over past and future times when considering stationary filtering. For electrons, $q=e$, the elementary charge. Choose the positive electric current direction to follow these transfers, so $I=q\lambda >0$ is its mean magnitude. If $N_T$ counts arrivals in a gate of duration $T>0$, thenThe gate's transferred electric charge is $Q_T=q{N}_T$, and its average electric current is ${\overline{i}}_T=Q_T/T$. Scaling the expected value and variance givesMore arrivals increase the absolute fluctuation but decrease its size relative to the mean. For an original count example with $\lambda T=4$, the mean count and its variance are both $4$, the standard deviation is $2$, and the relative standard deviation is $1/2$. The probability of no transfer is $e^{-4}\approx 0.01832$; the probability of four is $4^4{e}^{-4}/4!\approx 0.19537$. A constant arrival rate does not prescribe four arrivals in every gate.

These are fluctuations of a time average, not the variance of an ideal instantaneous electric current. A model of point transfers has arbitrarily sharp impulses; it acquires a finite measured variance only after specifying a gate or linear filter. Actual collection pulses have finite duration and electrical circuits have finite response times.

The Fano factor $F_T$ compares count variance with its mean over a specified gate. For a stationary unidirectional stream with fixed charge $q$, finite count second moments, and a generalized autocovariance with finite integral so that the long-gate limit exists,The zero-frequency relation concerns the continuous fluctuation spectrum, excluding the mean spectral line. It follows by writing $\mathrm{Var}(Q_T)$ as the double integral of the current autocovariance; for long $T$, its leading term is $T{S}_I^{(2)}(0)$. A finite-gate $F_T$, a frequency-dependent ratio $S_I^{(1)}(f)/(2qI)$, and a factor inferred from one filtered variance are not interchangeable. The last is weighted by the instrument's $|K(f){|}^2$.

Two original counting counterexamples show why electric charge discreteness is insufficient. In $n$ independent time slots, transfer one charge with probability $p$ in each slot: the binomial distribution gives mean $np$, variance $np(1-p)$ and Fano factor $1-p$. A random common origin for the slots removes a preferred time origin; the long-gate factor remains $1-p$. Alternatively, if independent Poisson process bursts each transfer exactly two charges, then $N_T=2M_T$ gives $\mathrm{Var}(N_T)=2\mathbb{E}[N_T]$ and factor $2$. Both count individual charges; only the first model allows at most one per slot, and the second groups them in pairs.

Consequently $2qI$ is a reference for independent unidirectional arrivals, not a universal noise law for all electrical conductors. Correlations, charge multiplication, and bidirectional transfer require a more specific model. For example, independent opposite streams with positive current magnitudes $I_{+}$ and $I_{-}$ have net mean $I=I_{+}-I_{-}$ but one-sided fluctuation spectrum $2q(I_{+}+I_{-})$. Using $2q|I|$ would miss their fluctuations when the net current is small. At thermal equilibrium, the full transport model must also respect thermal electrical noise; adding unrelated phenomenological formulas can double count the same underlying transfers.

For $m\ge 1$ independent valid null p-values, the displayed inequality follows by applying the Benjamini-Hochberg procedure with all nulls true. Its false discovery rate then equals the probability of at least one rejection. Valid means $\mathrm{Pr}(p_i\le t)\le t$ for $0\le t\le 1$; independence is a sufficient hypothesis, and arbitrary dependence is not covered by this proof.

A simple cooperative game is a monotone transferable utility game in which each coalition either loses, with value zero, or wins, with value one. In the usual nontrivial convention the empty coalition loses and the grand coalition wins. The Shapley value is then the probability that a player changes a losing coalition into a winning one when players enter in uniformly random order.

Here simple random sampling means a random unordered subset $A$ of exactly $n$ distinct labels from $\{1,\dots ,N\}$, with every size-$n$ subset equally likely. The sample size is fixed in advance, $1\le n\le N$, and sampling without replacement prevents duplicate units. This is SRSWOR; some texts also use "simple random sampling" for independent uniform draws with replacement, which is a different design.^[ref]

Every label has inclusion probability $n/N$, but equal inclusion probabilities alone are insufficient. For example, from six labels choose either $\{1,2,3\}$ or $\{4,5,6\}$ with probability $1/2$. Each label is included with probability $1/2$, yet eighteen of the twenty size-three subsets are impossible: this is a two-group cluster sampling design. Unequal probability sampling deliberately changes inclusion probabilities, often to improve precision for a particular estimand, and generally requires an appropriately weighted estimator rather than the plain sample mean.

Consider the original mathematical population with labelled values $(2,4,6,8,10,12)$, and let $n=2$. The fifteen equiprobable subsets, followed by their sample means, areTheir sample means sum to $105$, giving expected value $105/15=7$. Their squared deviations from $7$ sum to $70$, giving variance $70/15=14/3$. This is an enumeration over hypothetical selections from six stipulated numbers, not measured survey data.

With independent uniform sampling with replacement, the $6^2=36$ ordered outcomes each have probability $1/36$. The two orders $(2,4)$ and $(4,2)$ are distinct outcomes, and $(2,2)$ is now possible. Forgetting order does not make the resulting unordered multisets equiprobable: $\{2,4\}$ has probability $2/36$ but $\{2,2\}$ has probability $1/36$. The thirty-six sample means still have expected value $7$, but their squared deviations sum to $210$, giving variance $210/36=35/6$. Avoiding duplicate labels improves precision for this fixed population.

A simple random walk on a locally finite graph chooses each vertex in the current vertex's graph neighbourhood with equal probability, assuming there is at least one available neighbour. On the integer line there are two choices, one unit left or right, so each has probability $1/2$. This is the one-dimensional simple symmetric random walk. Terminology varies: a simple random walk on the integer line is also often allowed unequal step probabilities, with “symmetric” specifying the fair case.

For a simple symmetric random walk starting at $S_0=0$, each increment has the Rademacher distribution, with expected value zero and variance one. Thus linearity of expectation and the variance of a sum of independent random variables giveFor example, after $80$ steps the expected value is zero, the variance is $80$, and the standard deviation is $\sqrt{80}\approx 8.944$. Doubling the typical distance scale takes four times as many steps. This square-root growth explains why a random walk can wander far even when there is no preferred direction.

The colored curves are illustrative sample paths, not averages over all possible walks. The dashed curves show the expected values; the shaded bands show the expected value plus or minus one standard deviation at each time. These bands are a scale of spread, not confidence intervals, and give no guarantee that an entire sample path stays inside them.

Zero expected value does not make this a stationary process: its one-time probability distribution changes with $n$, as the increasing variance already proves. It does have stationary increments and independent increments, because blocks of fresh steps have the same probability distribution at any starting time and disjoint blocks use independent random variables. This separation of position from increment is one reason the random walk is such a useful first example of a stochastic process.

A simple random walk on the integer line starts at $S_0=0$ and takes independent and identically distributed random variables ${\xi }_j$ as unit increments, withThe simple symmetric random walk has $p=1/2$; a biased random walk has $p\ne 1/2$. At $p=0$ or $p=1$ every step is deterministic, giving $S_n=-n$ or $S_n=n$ respectively.

Count the rightward steps using indicator functions:Thus the endpoint is a shifted and rescaled random variable with a binomial distribution. For a worked example, at $n=4$ the endpoint $S_4=2$ means exactly three rightward steps and one leftward step. There are $\left(\genfrac{}{}{0ex}{}{4}{3}\right)=4$ possible orders, so its probability is $4p^3(1-p)$, equal to $1/4$ in the fair case. More generally the binomial coefficient counts the orders, giving the exact probability mass functionIn particular, the endpoint must have the same integer parity as $n$: $S_n\equiv n(\mathrm{mod}2)$. For $0<p<1$ the support of a probability distribution is exactly $\{-n,-n+2,\dots ,n\}$. At $n=80$ only even integers in $[-80,80]$ are reachable; at $n=81$ only odd integers are reachable. This restriction is easy to lose if one draws a smooth curve instead of discrete probability masses. The fair-walk counting argument is also presented in Grinstead and Snell, Introduction to Probability, section 12.1.

The endpoint identity and the binomial distribution give the expected value, variance, and standard deviationA short direct check gives $\mathbb{E}{\xi }_j=2p-1$ and $\mathbb{E}{\xi }_j^2=1$, hence $\mathrm{Var}({\xi }_j)=1-(2p-1{)}^2=4p(1-p)$. Then apply linearity of expectation and the variance of a sum of independent random variables.

The absolute width and the width relative to the number of steps answer different questions. In the fair case the standard deviation grows from $\sqrt{20}\approx 4.472$ to $\sqrt{80}\approx 8.944$, while ${\sigma }_n/n$ shrinks from $0.2236$ to $0.1118$. At $n=80,p=0.65$, the standard deviation is $\sqrt{72.8}\approx 8.532$, or $0.9539$ times the fair width, and ${\sigma }_n/n\approx 0.1067$. Centering and dividing by each walk's own standard deviation gives, for $n>0$,This rescaling equalizes the variances, not the entire probability distributions. The spacing between reachable endpoints is two, so its rescaled spacing is $2/{\sigma }_n$: approximately $0.4472$ for the fair $20$-step walk, $0.2236$ for the fair $80$-step walk, and $0.2344$ for the biased $80$-step walk.

The positions themselves are dependent. For $1\le m\le n$, $S_n=S_m+(S_n-S_m)$, where the second term is independent of $S_m$. Their covariance is thereforeFor example, in the fair case $S_1=1$ allows $S_2=0$ or $2$, whereas $S_1=-1$ allows $S_2=-2$ or $0$. Nevertheless the position process is a Markov chain: given its current position, the next fresh increment has no dependence on its past. Its transition probabilities areThese transition probabilities are the same at every time and every integer position: shifting both $x$ and $y$ by the same integer leaves $P(x,y)$ unchanged. Thus the Markov property and independent increments coexist with dependent positions.