Optimal stopping value function ($V(t,s)$)

For a Markov process with reward $f$, the value starting at time $t$ in state $s$ is the supremum of expected rewards over admissible stopping times. In finite discrete time it obeys $V(T,s)=f(s)$ and $V(t,s)=\max \{f(s),PV(t+1,\cdot )(s)\}$ for the transition operator $P$, wherever expectations are well-defined. Statewise absolute integrability of every remaining-horizon reward makes these values finite.

The transition operator of an additive random walk preserves convex functions: apply their convexity inequality to $x+\xi$ and $y+\xi$, then integrate. The pointwise maximum of convex functions preserves convexity too, so induction proves convexity of the Snell envelope Bellman functions. Expectations are allowed to be extended-valued until finiteness is justified. Integrability of rewards on a finite horizon implies finiteness of these functions at every state for $t\ge 1$, by integrability of translated convex random-walk rewards and the bound $W_t(x)\le {\sum }_{j=0}^{T-t}\mathbb{E}[f(x+Z_j{)}^{+}]$. At time zero only $X_0=0$ is observed; if $W_0$ is infinite elsewhere, its finite value at zero has a constant convex extension representing the same Snell envelope. Full statewise Bellman recursion at time zero requires an additional translated-integrability hypothesis.

Let $Z_j$ be the sum of $j$ independent identically distributed real increments. For a finite convex function $f$, integrability of $f(Z_j)$ and $f(Z_{j+1})$ implies integrability of $f(Z_j+x)$ for every fixed $x$. Its negative part is controlled by negative part of a finite convex function is Lipschitz. For $x>0$, if the increment law is unbounded above, take an independent increment $\eta$ with $p=\mathbb{P}(\eta \ge x)>0$. On this event convexity gives $f(Z_j+x{)}^{+}\le f(Z_j{)}^{+}+f(Z_j+\eta {)}^{+}$, so the expectation is bounded by $\mathbb{E}f(Z_j{)}^{+}+p^{-1}\mathbb{E}f(Z_{j+1}{)}^{+}$. If increments are bounded above by $M$, use $Z_j\le jM$ and bound the positive part by $f(Z_j{)}^{+}+f(jM+x{)}^{+}$. For $x<0$, use the symmetric lower-tail argument. This supplies translated integrability without assuming integrable increments or a growth bound on $f$.