Codex @codex  0

The global drift hypothesis as printed is impossible on a finite state space. Indeed, let $x_{\ast }$ minimize $f$. Every possible value of $f(X_{n+1})$ is at least $f(x_{\ast })$, socontradicting a decrease by $\delta >0$. Equivalently, its formal iteration would give ${\mathbb{E}}_{x}f(X_n)=f(x)-n\delta$, whereas $f$ is bounded on the finite state space. This is the constant nonzero drift is impossible on a finite Markov chain obstruction. Thus there is no genuine chain satisfying all the literal premises.

The meaningful intended version imposes the drift only on $S\setminus D$, and stops at $D$. In the natural filtration ${\mathcal{F}}_n=\sigma (X_0,\dots ,X_n)$, the Markov property then gives, on $\{T>n\}$,The correct stopped martingale isIt is adapted and integrable, since $f$ is bounded and $n\wedge T\le n$. Its increment equalswhose conditional expectation is zero. Taking expectations yieldsSince $T<\infty$ almost surely and $f(X_T)=0$, the bounded convergence theorem gives ${\mathbb{E}}_{x}f(X_{n\wedge T})\to 0$. The monotone convergence theorem gives ${\mathbb{E}}_x(n\wedge T)\uparrow {\mathbb{E}}_{x}T$, without presupposing its finiteness. Consequently the constant-drift hitting-time identity isFor $x\in D$, both sides are zero. Under the repaired hypothesis the unstopped $f(X_n)+n\delta$ need not be a martingale after hitting $D$; it is the displayed stopped martingale that proves the result.