Snell envelope

The finite-horizon Snell envelope of rewards $Z_n$ is defined backward by $V_N=Z_N$ and $V_n=\max \{Z_n,\mathbb{E}[V_{n+1}\mid {\mathcal{F}}_n]\}$. It is the least supermartingale dominating $Z$.

Let $Y_t=Y_0{\prod }_{j=1}^t{Z}_j$, where $Y_0>0$ is deterministic and the positive factors are independent and integrable. Use the natural filtration. Its finite-horizon Snell envelope isIndeed, backward induction and independence of random variables give $\mathbb{E}[U_{t+1}\mid {\mathcal{F}}_t]=Y_t{c}_{t+1}\mathbb{E}{Z}_{t+1}$; take the maximum with immediate exercise $Y_t$. The constants are finite and at least one. Equivalently, $c_t$ is the maximum of $1$ and the successive products ${\prod }_{j=t+1}^u\mathbb{E}{Z}_j$ for $t<u\le T$. If the factors have a common mean $\lambda >1$, waiting until maturity is optimal and $c_t={\lambda }^{T-t}$. Integrability and independence of random variables from the current information are essential to the finite deterministic coefficient conclusion.

Choosing a stopping time to maximize an expected reward. For an adapted integrable reward process over a finite discrete horizon, the Snell envelope computes the conditional optimal reward by comparing immediate exercise with continuation.

An optimal stopping problem in which observations update a posterior probability distribution over an unknown parameter. The decision state includes that posterior.

With independent uniform distributions on $[0,1]$ as sequential offers and a finite number of rounds, the continuation value with $j-1$ offers remaining is $v_{j-1}$. Accept the current offer when it reaches that threshold. Integrating the maximum of the offer and continuation value gives the recursion, by the Snell envelope principle.

At a regular exercise boundary for a diffusion optimal stopping problem, value matching is commonly accompanied by matching first derivatives. This makes the value $C^1$ across the boundary and eliminates a derivative-jump local time of a semimartingale term in the generalized Itô formula. It does not assert that second derivatives agree, or that smooth fit holds for every possible stopping problem.

For a Markov process with reward $f$, the value starting at time $t$ in state $s$ is the supremum of expected rewards over admissible stopping times. In finite discrete time it obeys $V(T,s)=f(s)$ and $V(t,s)=\max \{f(s),PV(t+1,\cdot )(s)\}$ for the transition operator $P$, wherever expectations are well-defined. Statewise absolute integrability of every remaining-horizon reward makes these values finite.

The transition operator of an additive random walk preserves convex functions: apply their convexity inequality to $x+\xi$ and $y+\xi$, then integrate. The pointwise maximum of convex functions preserves convexity too, so induction proves convexity of the Snell envelope Bellman functions. Expectations are allowed to be extended-valued until finiteness is justified. Integrability of rewards on a finite horizon implies finiteness of these functions at every state for $t\ge 1$, by integrability of translated convex random-walk rewards and the bound $W_t(x)\le {\sum }_{j=0}^{T-t}\mathbb{E}[f(x+Z_j{)}^{+}]$. At time zero only $X_0=0$ is observed; if $W_0$ is infinite elsewhere, its finite value at zero has a constant convex extension representing the same Snell envelope. Full statewise Bellman recursion at time zero requires an additional translated-integrability hypothesis.

Let $Z_j$ be the sum of $j$ independent identically distributed real increments. For a finite convex function $f$, integrability of $f(Z_j)$ and $f(Z_{j+1})$ implies integrability of $f(Z_j+x)$ for every fixed $x$. Its negative part is controlled by negative part of a finite convex function is Lipschitz. For $x>0$, if the increment law is unbounded above, take an independent increment $\eta$ with $p=\mathbb{P}(\eta \ge x)>0$. On this event convexity gives $f(Z_j+x{)}^{+}\le f(Z_j{)}^{+}+f(Z_j+\eta {)}^{+}$, so the expectation is bounded by $\mathbb{E}f(Z_j{)}^{+}+p^{-1}\mathbb{E}f(Z_{j+1}{)}^{+}$. If increments are bounded above by $M$, use $Z_j\le jM$ and bound the positive part by $f(Z_j{)}^{+}+f(jM+x{)}^{+}$. For $x<0$, use the symmetric lower-tail argument. This supplies translated integrability without assuming integrable increments or a growth bound on $f$.

An optimal stopping time attains the supremum of the expected reward over an admissible class of stopping times. For a finite-horizon Snell envelope, the first time at which its value equals the current reward is optimal.

For the Doob compensator of a Snell envelope, $(V_n-Z_n)(A_{n+1}-A_n)=0$: compensation grows only on the stopping region $V_n=Z_n$.

The first time the Snell-envelope compensator is about to become positive is an optimal stopping time; before it the martingale part equals both value and reward at stopping.