Optimal stopping (${\sup }_{\tau }\mathbb{E}[Z_{\tau }]$)

Choosing a stopping time to maximize an expected reward. For an adapted integrable reward process over a finite discrete horizon, the Snell envelope computes the conditional optimal reward by comparing immediate exercise with continuation.

An optimal stopping problem in which observations update a posterior probability distribution over an unknown parameter. The decision state includes that posterior.

With independent uniform distributions on $[0,1]$ as sequential offers and a finite number of rounds, the continuation value with $j-1$ offers remaining is $v_{j-1}$. Accept the current offer when it reaches that threshold. Integrating the maximum of the offer and continuation value gives the recursion, by the Snell envelope principle.

At a regular exercise boundary for a diffusion optimal stopping problem, value matching is commonly accompanied by matching first derivatives. This makes the value $C^1$ across the boundary and eliminates a derivative-jump local time of a semimartingale term in the generalized Itô formula. It does not assert that second derivatives agree, or that smooth fit holds for every possible stopping problem.

For a Markov process with reward $f$, the value starting at time $t$ in state $s$ is the supremum of expected rewards over admissible stopping times. In finite discrete time it obeys $V(T,s)=f(s)$ and $V(t,s)=\max \{f(s),PV(t+1,\cdot )(s)\}$ for the transition operator $P$, wherever expectations are well-defined. Statewise absolute integrability of every remaining-horizon reward makes these values finite.

The transition operator of an additive random walk preserves convex functions: apply their convexity inequality to $x+\xi$ and $y+\xi$, then integrate. The pointwise maximum of convex functions preserves convexity too, so induction proves convexity of the Snell envelope Bellman functions. Expectations are allowed to be extended-valued until finiteness is justified. Integrability of rewards on a finite horizon implies finiteness of these functions at every state for $t\ge 1$, by integrability of translated convex random-walk rewards and the bound $W_t(x)\le {\sum }_{j=0}^{T-t}\mathbb{E}[f(x+Z_j{)}^{+}]$. At time zero only $X_0=0$ is observed; if $W_0$ is infinite elsewhere, its finite value at zero has a constant convex extension representing the same Snell envelope. Full statewise Bellman recursion at time zero requires an additional translated-integrability hypothesis.

Let $Z_j$ be the sum of $j$ independent identically distributed real increments. For a finite convex function $f$, integrability of $f(Z_j)$ and $f(Z_{j+1})$ implies integrability of $f(Z_j+x)$ for every fixed $x$. Its negative part is controlled by negative part of a finite convex function is Lipschitz. For $x>0$, if the increment law is unbounded above, take an independent increment $\eta$ with $p=\mathbb{P}(\eta \ge x)>0$. On this event convexity gives $f(Z_j+x{)}^{+}\le f(Z_j{)}^{+}+f(Z_j+\eta {)}^{+}$, so the expectation is bounded by $\mathbb{E}f(Z_j{)}^{+}+p^{-1}\mathbb{E}f(Z_{j+1}{)}^{+}$. If increments are bounded above by $M$, use $Z_j\le jM$ and bound the positive part by $f(Z_j{)}^{+}+f(jM+x{)}^{+}$. For $x<0$, use the symmetric lower-tail argument. This supplies translated integrability without assuming integrable increments or a growth bound on $f$.