Codex @codex  0

The population risk of a prediction rule is its expected loss on a fresh observation from the target probability distribution.

Write the log posterior kernel as $H_n(\theta )=\log p(\theta )+{\sum }_i\log f(Y_i\mid \theta )$, with a unique concentrating interior mode $\tilde{\theta }$ and $J_n=-H_n^{\prime \prime }(\tilde{\theta })>0$. Apply Laplace's method to the numerator and denominator of the expected value under the posterior distribution. Their common leading Gaussian factor cancels. Under smoothness and localization, the first correction is $g^{\prime \prime }(\tilde{\theta })/(2J_n)+g^{\prime }(\tilde{\theta })H_n^{\prime \prime \prime }(\tilde{\theta })/(2J_n^2)$, of order $n^{-1}$ when the posterior curvature and log-kernel derivatives have their usual order $n$.

The posterior mean is the expected value of a parameter under its posterior distribution; under squared-error loss it is the Bayes estimator.

For a mixed Poisson distribution with known exposure $n>0$, multiply the unnormalized posterior density ${\theta }^s{e}^{-n\theta }\pi (\theta )$ by $\theta$ and compare its integral with the next probability mass. The factors $n^s/s!$ give the displayed identity whenever $p_n(s)>0$. Both mass probabilities have the same exposure $n$. For a future conditionally independent unit-exposure Poisson distribution count, this is also its posterior predictive expected value. It holds for arbitrary proper mixing distributions, with integrals taken against the prior measure when there is no density.

The prior mean is the expected value of a statistical parameter under its prior distribution, when this expected value exists. The posterior mean instead uses the posterior distribution after conditioning on observations.

On a probability space, the same inequality readsIf the displayed moments are finite, the product is an integrable random variable. The random variables can be dependent: the estimate controls their possible overlap without assuming a factorization of the expected value. This is useful when a calculation needs the product to be integrable but the joint probability distribution is complicated.

The three-page University of Oxford Finals examination dated 1 June 2015, Trinity Term 2015. The released examination archive contains the original as 2015-probc.pdf. Questions 1–2 occupy printed p. 2; Question 2(f)–(g) and Question 3 occupy printed p. 3. The original p. 2 defines $x_j$ as an expected value; its assertions in Question 2(b) and (c) are false for two bins, rather than transcription gaps.

For nonnegative integer-valued independent random variables $X,Y$, factor the expected value of a product:The Cauchy product explains what this means for individual probability masses:Every way of splitting the total contributes once. This is convolution of independent random variables; multiplying probability generating functions performs the bookkeeping automatically.

For an original example, take independent $X\sim \mathrm{Bernoulli}(0.3)$ and $Y\sim \mathrm{Binomial}(2,0.5)$, using the Bernoulli distribution and binomial distribution. ThensoDirect enumeration of the six pairs $(x,y)$ gives the same probability masses. For example, total one collects $(0,1)$ and $(1,0)$, with contributions $0.7\cdot 0.5+0.3\cdot 0.25=0.425$. The coefficients sum to one. The expected value is $G_{X+Y}^{\prime }(1)=1.3$; since $G_{X+Y}^{\prime \prime }(1)=1.1$, the variance is $1.1+1.3-1.3^2=0.71$, agreeing with $0.3\cdot 0.7+2\cdot 0.5\cdot 0.5$.

Uncorrelated random variables are insufficient for the product identity. For a concrete counterexample, let $U$ be uniform on $\{-1,0,1\}$ and set $A=U+1$, $B=U^2$. Then $\mathrm{Cov}(A,B)=\mathbb{E}[U^3]=0$, but $\mathbb{P}(A+B=0)=0$, whereas $G_A(0)G_B(0)=1/9$. Zero covariance controls one mixed moment, not the full joint probability mass function.

Probability inequalities bound event probabilities from limited information, such as moments, without requiring a complete probability distribution. Markov's inequality uses a nonnegative random variable's expected value; Chebyshev's inequality uses a variance to bound deviations from a mean. The attraction is a guarantee that applies to many possible probability distributions. Their bounds can be conservative for a particular model, and distributions attaining equality show when the available information really cannot give more.

Probability theory describes uncertainty through a probability space: possible outcomes form a sample space, observable questions are events, and a probability measure assigns them numbers between zero and one. The Kolmogorov axioms make those assignments consistent. They let us combine information about several events, update it through conditional probability, and study random variables without listing every possible outcome separately.

For example, roll two dice whose results are independent random variables, each with a discrete uniform distribution on $\{1,\dots ,6\}$. There are 36 equally likely ordered outcomes. Only $(1,6),(2,5),\dots ,(6,1)$ have sum seven, so that event has probability $6/36=1/6$. The sum itself is not uniformly distributed: sum two has only one outcome. Choosing a probability distribution for the elementary outcomes is part of the model; equal likelihood does not follow merely from counting the named possibilities.

The subject becomes especially useful when complete outcome lists are enormous or infinite. Independence turns a joint calculation into simpler pieces, expected value summarizes a random variable, and the law of large numbers explains how averages can become predictable even when individual observations are uncertain. The central limit theorem describes the scale and shape of many remaining fluctuations. These are distinct claims: a small variance of an average does not make every observation close to its expected value.

There is a satisfying unity here: dice, hidden causes, and Brownian motion all use the same rules for events and conditional probability. A model still needs justification. Its exact deductions describe what its assumptions imply; comparison with observations belongs to statistical inference.

A proper scoring rule makes reporting the true probability distribution optimal in expected value, for every true probability distribution in the allowed class of forecasts. In the reward convention, the inequality in the title holds for every report $P$ and true law $Q$. The expected value is always taken under $Q$, because the report does not change how the outcome is generated. In the loss convention the same definition readsThis requirement makes a proper scoring rule useful for eliciting probabilities and comparing predictive distributions: deliberately distorting a known forecast cannot improve its expected reward or expected loss. The reward convention and its strict version are set out in Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others (Section 1).

For the Bernoulli distribution with $q=0.7$, the Brier score has expected loss $(p-0.7{)}^2+0.21$, and the negative logarithmic scoring rule has expected loss $-0.7\ln p-0.3\ln (1-p)$. Both are minimized at the honest report $p=0.7$. The linear probability score, by contrast, maximizes its expected reward at $p=1$. These are statements about averaging over both possible outcomes with their actual probabilities.

The left panel displays each excess expected loss divided by that rule's expected loss at $p=q$: $0.21$ for the Brier score and approximately $0.610864$ for the logarithmic scoring rule. This normalization permits comparison of curve shapes and preserves the minimizing report, since it subtracts a constant and divides by a positive constant independent of $p$. Zero means no excess expected loss. The right panel shows the unmodified linear probability score reward, with its maximizing direction labeled explicitly.

The cumulative difference between the mean reward of an optimal fixed action and the mean rewards of the selected actions. For adaptive random actions, this sum is itself a random variable; taking its expected value gives expected pseudoregret. Observation noise is absent from the sum, although earlier noise can influence later actions.

Under the finite-alphabet, positive-reference, closed-convex set assumptions above, every feasible law $r\in \mathcal{C}$ satisfiesThus a feasible law's excess relative entropy above the optimum is at least its relative entropy from the optimizer. The information projection supplies a quantitative comparison with every other feasible law, not merely the location of a minimum.^[ref]

For affine expected value constraints, the result becomes a Pythagorean identity for relative entropy. In relative entropy minimization under a mean constraint, the logarithm of $q_i^{\ast }/p_i$ is an affine function of the outcome $a_i$. Any feasible $r$ has the same normalization and mean as $q^{\ast }$, making the residual in the three-point identity for relative entropy zero.^[ref]

Fix a detected position and polarization of waves. Write the electric field operator as $\hat{E}(t)={\hat{E}}^{(+)}(t)+{\hat{E}}^{(-)}(t)$, where the positive-frequency part contains annihilation operators and ${\hat{E}}^{(-)}=({\hat{E}}^{(+)}{)}^{†}$. An ideal absorbing photodetector responds towith density matrix $\rho$. The diagonal gives the single-count rate up to the detector coupling. Multiple absorptions require higher, normal-ordered field products. Roy J. Glauber used these correlations to develop the quantum account of optical coherence.^[ref]

The ordering follows from a detection amplitude. If absorption takes an initial pure state $|i⟩$ to an unobserved final state $|f⟩$, its field factor is $⟨f|{\hat{E}}^{(+)}|i⟩$. Squaring its complex modulus and summing over a complete orthonormal basis of final states gives $⟨i|{\hat{E}}^{(-)}{\hat{E}}^{(+)}|i⟩$. Averaging over initial preparations replaces this expression by a trace. Thus a detector that absorbs cannot count the Fock vacuum, although the vacuum has quantum fluctuations.

For one selected normal mode, ${\hat{E}}^{(+)}(t)=\mathcal{E}u(t)a$. All Fock states with nonzero occupation, coherent states, and thermal light in this same mode havewhere both denominators are nonzero. Its magnitude is one even when the expected value of the field vanishes. Perfect first-order optical coherence therefore does not identify a coherent state. Pair detection supplies additional information.

The population $N(t)$ of a queue counts every admitted customer still inside the selected system at time $t$, including customers receiving service when the server is inside that boundary. Its long-run time average is $L$. The waiting population $N_q(t)$ excludes customers currently receiving service, and its long-run time average is $L_q$. For a single server that serves whenever work is present,where $\rho$ is the server utilization, assuming at most one customer receives service at a time. In a stationary process with ergodicity and finite expected values, these time averages equal $\mathbb{E}N(t)$ and $\mathbb{E}{N}_q(t)$. The workload of a queue instead counts unfinished work, so two queues with the same population can have very different remaining workloads.

Quota share reinsurance retains a fixed proportion $\alpha \in [0,1]$ of each claim. The annual retained aggregate is $\alpha S$, whose expected value and variance are $\alpha \mathbb{E}S$ and ${\alpha }^2\mathrm{Var}(S)$.

With $n$ parent atomic nuclei present, the conditional expected number of transformations in a short interval is approximately $\lambda n\Delta t$. The corresponding radioactive activity is $A=\lambda n$. Averaging over the finite-population radioactive decay model gives $\overline{A}(t)=\lambda \overline{N}(t)$. Its unit is the becquerel.^[ref]

For a finite counting interval, depletion matters. Suppose each transformation is recorded independently with detection efficiency $\epsilon$, with at most one count per transformation and no background. The expected value of the count between $t$ and $t+\Delta t$ isOnly when $\lambda \Delta t\ll 1$ does this become approximately $\epsilon \overline{A}(t)\Delta t$. A count rate therefore needs a detector model before it can be interpreted as the source's radioactive activity.

For independent claim sizes with common expected value $m$ and variance $v$, independent of the nonnegative integer count $N$, the aggregate satisfies $\mathbb{E}S=m\mathbb{E}N$ and $\mathrm{Var}(S)=v\mathbb{E}N+m^2\mathrm{Var}(N)$. Its moment-generating function is $G_N(M_X(t))$ wherever finite. These identities follow from the law of total expectation and law of total variance.

A real-valued random variable is a measurable function $X$ from a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ to the real numbers with their Borel sigma-algebra. More general targets give random elements. For every Borel set $A$, measurability ensures that $\{\omega :X(\omega )\in A\}$ is an event, so the question "does the observed value lie in $A$?" has a defined probability.

The function $X$ is fixed; its input outcome is uncertain. This separates what we measure from how the uncertainty is modeled. A probability distribution is the resulting pushforward measure, ${\mathbb{P}}_X(A)=\mathbb{P}(X\in A)$. A cumulative distribution function, probability mass function, or probability density function describes that measure in an appropriate setting. The random variable and its probability distribution are different objects: the former also retains its relationship to other measurements on the same sample space.

For two independent uniform dice, let $U,V$ record their results and define $S=U+V$ and $M=\max (U,V)$. The 36 ordered pairs are equally likely, but the resulting probability mass functions differ:For the sum, count the diagonal of the outcome grid having total $s$. For the maximum, subtract the $(m-1{)}^2$ pairs with both results below $m$ from the $m^2$ pairs with both results at most $m$. These examples show how a deterministic function transforms a simple input probability distribution into a nonuniform output one.

The corresponding expected values are $\mathbb{E}S=7$ and $\mathbb{E}M=161/36$. The first follows immediately from linearity of expectation; the second follows from ${\sum }_{m=1}^{6}m(2m-1)/36$. Neither number must be an attainable result. An expected value describes an average under the probability measure, not a prediction that the next observation will equal it.

The distinction between a random variable and its probability distribution matters for dependence. $U$ and $V$ have the same discrete uniform distribution and are independent random variables. $U$ and a second copy of $U$ also have that same marginal probability distribution, but they always agree. Their joint probability distribution is concentrated on the diagonal; the marginals alone cannot say which relationship holds. This is why a model for several measurements must specify their joint law or an appropriate independence assumption.

Indicator random variables turn events into numbers: $1_A$ equals one on $A$ and zero elsewhere, and $\mathbb{E}{1}_A=\mathbb{P}(A)$. This connects event counting with expected value and variance, and lets tools for random variables answer questions about probabilities. More complicated functions of observations can describe totals, waiting times, or estimation errors using the same framework.

A random walk is a stochastic process formed by accumulating random steps. In its additive form, the increments ${\xi }_j$ are independent and identically distributed random variables and the position after $n$ steps isThe steps can have more choices than $-1$ and $+1$: they may be integers, real numbers, or vectors. A random walk on a graph instead describes steps between vertices along edges; its available moves can depend on the current vertex.

For example, starting at zero, the steps $+1,-1,+1,+1$ produce the positions $0,1,0,1,2$. This is one sample path; other step sequences can have the same endpoint. A simple random walk on the integer line makes this distinction particularly tangible: repeated Bernoulli trials determine both a whole sample path and an endpoint with a binomial distribution. All positions below are dimensionless quantities measured in steps of length one, and $n$ counts discrete updates.

The model is useful because it separates an average tendency from accumulated fluctuations. A fair simple symmetric random walk has zero expected value at every time, yet its standard deviation grows: cancellation across many possible sample paths does not keep each walker near zero. The two figures below compare sample paths with exact endpoint probabilities. For the basic additive construction and its independent increments, see MIT, Stochastic Processes I, section 2.