= Solution
\b[The global drift hypothesis as printed is impossible on a finite state space.] Indeed, let $x_*$ minimize $f$. Every possible value of $f(X_{n+1})$ is at least $f(x_*)$, so
$$
\mathbb E[f(X_{n+1})\mid X_n=x_*]\geq f(x_*),
$$
contradicting a decrease by $\delta>0$. Equivalently, its formal iteration would give $\mathbb E_xf(X_n)=f(x)-n\delta$, whereas $f$ is bounded on the finite state space. This is the <constant nonzero drift is impossible on a finite Markov chain> obstruction. Thus there is no genuine chain satisfying all the literal premises.
The meaningful intended version imposes the drift only on $S\setminus D$, and stops at $D$. In the <natural filtration> $\mathcal F_n=\sigma(X_0,\ldots,X_n)$, the <Markov property> then gives, on $\{T>n\}$,
$$
\mathbb E[f(X_{n+1})-f(X_n)\mid\mathcal F_n]=-\delta.
$$
The correct <stopped martingale> is
$$
\boxed{\widetilde Y_n=f(X_{n\wedge T})+\delta(n\wedge T).}
$$
It is adapted and integrable, since $f$ is bounded and $n\wedge T\leq n$. Its increment equals
$$
\widetilde Y_{n+1}-\widetilde Y_n
=\mathbf1_{\{T>n\}}\bigl(f(X_{n+1})-f(X_n)+\delta\bigr),
$$
whose <conditional expectation> is zero. Taking <expectations> yields
$$
\mathbb E_x f(X_{n\wedge T})+\delta\mathbb E_x(n\wedge T)=f(x).
$$
Since $T<\infty$ <almost surely> and $f(X_T)=0$, the <bounded convergence theorem> gives $\mathbb E_x f(X_{n\wedge T})\to0$. The <monotone convergence theorem> gives $\mathbb E_x(n\wedge T)\uparrow\mathbb E_xT$, without presupposing its finiteness. Consequently the <constant-drift hitting-time identity> is
$$
\boxed{\mathbb E_xT=\frac{f(x)}\delta<\infty.}
$$
For $x\in D$, both sides are zero. Under the repaired hypothesis the unstopped $f(X_n)+n\delta$ need not be a <martingale> after hitting $D$; it is the displayed <stopped martingale> that proves the result.
Back to article page
