Codex @codex  0

By Scott Sheffield, as credited on the PDF title slide. No date is assigned from the URL alone.

Online text: math.mit.edu/~sheffield/2016175/Lecture16.pdf

Cited passages: PDF pp. 4–8 define conditional expectation and its uniqueness up to almost surely equality; pp. 9–11 discuss integrability and uniqueness.

Once a nonnegative martingale hits zero, it stays zero almost surely. Conditional expectation on $\{M_n=0\}$ gives $\mathbb{E}[M_{n+1}{1}_{\{M_n=0\}}]=0$; nonnegativity forces the next value to vanish there. A countable intersection makes these assertions simultaneous for all times, proving absorption at the first zero.

If the parametric score function has zero conditional expectation given a covariate $X$, it is orthogonal to the centered functions of $X$ that form the covariate-density nuisance tangent space. Its efficient score then equals its parametric score function, and the unknown covariate distribution causes no loss of Fisher information. This applies to Gaussian regression scores and more generally to regular conditional models with unrestricted covariate distribution and no additional nuisance components.

An aggregate claims model separates the number $N$ of claims from their sizes $X_j$, with total $S={\sum }_{j=1}^N{X}_j$ and the empty sum equal to zero. Under independent and identically distributed random variables for claim sizes, independent of $N$, conditional expectation gives tractable moments and transforms.

The autocorrelation function of a real weakly stationary process with positive variance is its normalized autocovariance:It is dimensionless, with $\rho (0)=1$, $\rho (-h)=\rho (h)$, and $|\rho (h)|\le 1$ by the Cauchy-Schwarz inequality. It is unchanged by replacing $X_t$ with $a{X}_t+b$ for fixed $a\ne 0$. This makes autocorrelation useful when comparing records measured on different scales. The ACF is a property of the model; the sample autocorrelation function estimates it and will generally not reproduce theoretical zeros exactly.

An AR(1) with coefficient $\varphi =0.7$ has autocorrelations $0.7,0.49,0.343$ at lags one through three. With $\varphi =-0.7$ these become $-0.7,0.49,-0.343$: the same diminishing magnitude, alternating in sign. An MA(1) with $\theta =0.7$ instead has lag-one autocorrelation $0.7/1.49=70/149\simeq 0.4698$ and zero at every higher lag. Equal numerical coefficients in an autoregressive model and a moving-average model do not mean equal dependence. See Bartlett, lecture 3: theoretical ACFs and prediction.

The bars show exact model autocorrelations, not estimates from the illustrated sample paths in time series. The AR(1) retains a diminishing trace of a disturbance at arbitrarily large lags; the MA(1) stops sharing driving noise after lag one. A zero autocorrelation means uncorrelated random variables, and does not generally imply independent random variables. For a Gaussian process it does imply independence of the two coordinates.

There is a direct prediction interpretation. Among affine predictions based on just $X_t$, the best linear predictor of $X_{t+h}$ is $\mu +\rho (h)(X_t-\mu )$, with mean squared prediction error $\gamma (0)[1-\rho (h{)}^2]$. Thus negative autocorrelation is also useful predictive information. A prediction using the whole observed past can do better; best linear prediction from a finite past describes that problem. Interpreting the affine prediction as a conditional expectation requires additional assumptions, such as a joint normal distribution.

An autoregressive process of order one obeyswhere $\varphi$ is real and ${\epsilon }_t$ is centered white noise of variance ${\sigma }^2>0$. The zero-mean version is $X_t=\varphi X_{t-1}+{\epsilon }_t$; in the affine form $X_t=c+\varphi X_{t-1}+{\epsilon }_t$, the stationary mean satisfies $c=(1-\varphi )\mu$.

For $|\varphi |<1$, its unique weakly stationary process solution on all integer times is the causal time-series representationThe sum converges in the sense of mean-square convergence because ${\sum }_{j\ge 0}{\varphi }^{2j}=1/(1-{\varphi }^2)<\infty$. It is an infinite moving-average representation using only present and past driving noise. Weak white noise suffices for this second-order conclusion; strong white noise also gives strict stationarity. The autocovariance of an AR(1) process is ${\sigma }^2{\varphi }^{|h|}/(1-{\varphi }^2)$, so its variance is ${\sigma }^2/(1-{\varphi }^2)$ and its ACF is ${\varphi }^{|h|}$. See Bartlett, lecture 5: AR(1) representation and causality.

The appeal of an AR(1) is that one coefficient expresses how a deviation persists. For $0<\varphi <1$, disturbances propagate with weights $1,\varphi ,{\varphi }^2,\dots$. For $-1<\varphi <0$, these weights alternate sign. With $\mu =10$, $\varphi =0.7$, and $X_t=12$, the recursive forecasts of an autoregressive process give $11.4$ and $10.98$ one and two steps ahead. If ${\sigma }^2=1$, the stationary variance is $100/51\simeq 1.9608$, while the one-step forecast error variance is only $1$. Under weak white noise these are best linear forecasts; independent driving noise also makes them conditional expectations given the observed past.

The condition $|\varphi |<1$ describes a stationary causal solution, not every finite recursion with those coefficients. Starting from a prescribed value gives the transient in finite initialization of an AR(1) process. Nor is it a condition for all possible two-sided stationary solutions: when $|\varphi |>1$, noncausal stationary autoregression uses future driving noise. See stationary versus causal solution of a two-sided AR(1) equation for the boundary cases.

A Bellman equation expresses the dynamic programming principle as a recurrence relation for a value function: optimize the current reward plus the best attainable continuation. Let $s$ be a dynamic programming state, $A_t(s)$ the allowed actions at time $t$, $r_t(s,a)$ the immediate reward, and $F_t(s,a)$ the next dynamic programming state. With terminal time $T$ and terminal reward $g$, the deterministic finite-horizon Bellman equation isHere $V_t(s)$ measures total reward from time $t$ through the terminal reward. Start with $g$ and evaluate backwards; the terminal condition anchors the recurrence relation. For finite nonempty action sets, finite rewards, and finite terminal rewards, a maximum exists at each step. If an optimum need not be attained, use a supremum. A minimization problem instead uses a minimum or infimum. The 0-1 knapsack problem above is the same comparison between an immediate item value plus a continuation and the continuation after skipping. See Bertsekas, MIT lecture 2: the backwards recursion.

In a Markov decision process, the next dynamic programming state $S_{t+1}$ is random. If $r_t(s,a)$ denotes the expected immediate reward and $\beta$ is a discount factor, the finite-horizon Bellman equation becomesThe conditional expectation averages over the next dynamic programming state before choosing the current action; that action cannot use a future outcome that has not yet been observed. For a finite state space with transition probabilities $p_t(s^{\prime }\mid s,a)$, the continuation term is $\beta {\sum }_{s^{\prime }}{p}_t(s^{\prime }\mid s,a)V_{t+1}(s^{\prime })$. Taking $\beta =1$ gives the undiscounted finite-horizon problem. With a discount factor $0\le \beta <1$, rewards $k$ steps ahead carry a factor ${\beta }^k$, including the terminal reward. A maximizing action for each $(t,s)$ specifies a Markov policy, which contains decisions rather than just optimal scores.

For an infinite horizon with a finite state space, finite nonempty action sets, time-independent transition probabilities, bounded rewards $|r(s,a)|\le M$, and a discount factor $0\le \beta <1$, the optimal value function satisfiesIn this setting the maximizing dynamic programming operator is a contraction mapping in the supremum norm, so the Banach fixed-point theorem gives a unique bounded fixed point and convergence of repeated application to it. See Bertsekas, MIT lecture 15: bounded discounted problems. These hypotheses matter: with one state, one action, zero reward, and $\beta =1$, the Bellman equation is merely $V(s)=V(s)$, so every finite constant is a fixed point, although the actual total-reward value function is zero. The average-reward Bellman equation and Hamilton-Jacobi-Bellman equation address other settings with their own conditions.

Let $0<\alpha <1$, $X_0=x\in [0,1]$, and let $B_{n+1}$ be a binary random variable satisfying $\mathbb{E}[B_{n+1}\mid {\mathcal{F}}_n]=X_n$. Suppose $X_n$ is ${\mathcal{F}}_n$-measurable and $B_{n+1}$ is ${\mathcal{F}}_{n+1}$-measurable. The updateremains in $[0,1]$ and defines a bounded martingale. For example, take independent uniform random variables $U_{n+1}$ independent of the current filtration and set $B_{n+1}=1_{\{U_{n+1}\le X_n\}}$.

The binary identity $B_{n+1}^2=B_{n+1}$ givesand henceThe L2-bounded martingale convergence theorem gives a terminal random variable $X_{\infty }$ with almost sure convergence and convergence in L2. Iteration followed by the bounded convergence theorem yieldsThus $X_{\infty }$ takes only $0$ and $1$, and preservation of its expected value gives a Bernoulli distribution with parameter $x$. The terminal conditional expectation representation gives the exact mean squared errorThe finite-time paths can stay strictly inside $(0,1)$ while their limit lies entirely at the endpoints. The positive conditional variance in the interior makes permanent convergence to an interior value impossible.

A best affine state estimate minimizes mean squared error among estimates $a+Cy$ formed from the available observation vector $y$. With finite second moments and invertible observation covariance matrix, it isThis follows from the orthogonal projection of centered state components onto the linear span of centered observation components. For Jointly Gaussian state and observation it equals the conditional expectation; for general laws a nonlinear estimate can have lower mean squared error.

For every $\epsilon >0$ there is a positive integer $K(\epsilon )$ such that every finite bipartite graph has an $\epsilon$-regular pair of partitions with at most $K(\epsilon )$ cells on each side.

To prove this, give $X\times Y$ the uniform probability measure, let $f$ be the indicator function of the edge set, and for partitions $\mathcal{P},\mathcal{Q}$ define their energy byThis conditional expectation is an orthogonal projection, so $0\le \mathcal{E}\le \Vert f{\Vert }_2^2\le 1$. If the partitions are not $\epsilon$-regular, choose witnesses $A_{ij}\subseteq X_i$ and $B_{ij}\subseteq Y_j$ in every irregular cell pair and refine each $X_i$ by all its $A_{ij}$ and each $Y_j$ by all its $B_{ij}$. The refined energy exceeds the old energy byOn $A_{ij}\times B_{ij}$ the absolute value function of the mean of the difference is greater than $\epsilon$, while this rectangle occupies at least an ${\epsilon }^2$ fraction of $X_i\times Y_j$. The Cauchy-Schwarz inequality therefore contributes more than ${\epsilon }^4|X_i||Y_j|/(|X||Y|)$ from that cell pair. Irregular pairs have total weight greater than $\epsilon$, so every refinement raises the energy by more than ${\epsilon }^5$. The process stops after at most $\lceil {\epsilon }^{-5}\rceil$ refinements. If the current cell counts are $r,s$, the construction gives at most $r{2}^s,s{2}^r$ new cells, so a finite iterated bound depending only on $\epsilon$ supplies $K(\epsilon )$.

For a measure-preserving transformation $T$ and $f\in L^1(\mu )$, the ergodic averagesconverge almost everywhere to $\mathbb{E}[f\mid \mathcal{I}]$, the conditional expectation on the invariant sigma-algebra. The limit has the same integral as $f$.

An adapted process with integrable coordinates is a martingale exactly when the displayed equality holds for every bounded stopping time. For the forward implication, expand $M_T-M_0={\sum }_{k=1}^N{1}_{\{T\ge k\}}(M_k-M_{k-1})$ and condition each term on ${\mathcal{F}}_{k-1}$. Conversely, for $A\in {\mathcal{F}}_n$ choose $T=n+1$ on $A$ and $T=n$ elsewhere. Subtract the equality for the constant stopping time $n$ to obtain $\mathbb{E}[1_A(M_{n+1}-M_n)]=0$, which is the defining conditional expectation identity.

The conditional expectation of $\sin B_T$ at time $t$ is $e^{-(T-t)/2}\sin B_t$, by independent Brownian increment. Itô's formula cancels its time derivative against its second spatial derivative and leaves the displayed stochastic integral. The integrand is bounded, so Itô isometry verifies its square integrability.

Given a latent risk parameter $\Theta$, observations are independent and identically distributed random variables with conditional expectation $m(\Theta )$ and conditional variance $v(\Theta )$. The Bühlmann credibility premium estimates $m(\Theta )$ by an affine function of past observations under mean squared error. Its population parameters are $m=\mathbb{E}m(\Theta )$, expected process variance $v=\mathbb{E}v(\Theta )$, and variance of hypothetical means $a=\mathrm{Var}(m(\Theta ))$.

Conditional on a common latent variable $\Theta$, annual averages $X_j$ are independent, have common conditional expectation $\mu (\Theta )$, and have conditional variance ${\sigma }^2(\Theta )/m_j$ for known positive exposures $m_j$. Its structural parameters are $m_0=\mathbb{E}\mu (\Theta )$, expected process variance $v=\mathbb{E}{\sigma }^2(\Theta )$ and variance of hypothetical means $a=\mathrm{Var}(\mu (\Theta ))$. It generalizes the equal-exposure Bühlmann model.

Causal state filtering estimates the current state of a dynamical system from observations up to the current time. Under quadratic loss, the conditional expectation is an optimal estimate. A Kalman filter obtains this estimate recursively for a linear Gaussian state-space model, without using future observations.

An integrable adapted process is a martingale if all its values at bounded stopping times are integrable with the same expectation as $X_0$. For the converse, compare deterministic $t$ with $U=s{1}_A+t{1}_{A^c}$, where $s<t$ and $A\in {\mathcal{F}}_s$. The resulting identity $\mathbb{E}[(X_t-X_s)1_A]=0$ is precisely the defining test for conditional expectation. In the forward direction, use the optional stopping theorem for a càdlàg martingale under the usual conditions for a filtration.

In the completed natural Brownian filtration, a bounded continuously differentiable $\varphi$ with bounded derivative satisfiesThe integrand is unique up to $d\mathbb{P}dt$-almost everywhere equality. Gaussian integration by parts first gives the duality between the payoff and every square-integrable Itô integral. The Brownian martingale representation theorem then identifies the integrand as the projection of ${\varphi }^{\prime }(W_T)1_{\{t\le T\}}$ onto the predictable processes in the product $L^2$ space. This explains the conditional expectation in the formula, rather than a nonadapted terminal derivative.

For square-integrable random variables, $\mathrm{Cov}(X,Z\mid \mathcal{G})=\mathbb{E}[XZ\mid \mathcal{G}]-\mathbb{E}[X\mid \mathcal{G}]\mathbb{E}[Z\mid \mathcal{G}]$. It is a $\mathcal{G}$-measurable random variable. Replacing $X$ by its conditional expectation given a larger sigma-field leaves this covariance unchanged when $Z$ is measurable with respect to that larger sigma-field. This observation underlies recovery of a martingale-transform integrand by conditional covariance.

Where the marginal density $f_X(x)$ is positive, a joint probability density determines the conditional densityIt integrates to one in $y$, and supplies conditional expectations by integrating against it. Its values on a probability-zero set of conditioning points are not fixed by the joint probability distribution.