Discrete-time Markov chain martingale characterization ($M_n^f=f(X_n)-f(X_0)-{\sum }_{j<n}(Pf-f)(X_j)$)

An adapted process on a finite or countable state space has the Markov chain transition matrix $P$ exactly when the displayed process is a martingale for every bounded test function $f$. The forward implication follows by taking the conditional expectation of its increment. Conversely, the martingale property gives $\mathbb{E}[f(X_{n+1})\mid {\mathcal{F}}_n]=(Pf)(X_n)$; testing individual-state indicators recovers the entire conditional transition law. Stopping these martingales turns equations $(P-I)h=0$ and $(P-I)v=-1$ into hitting probabilities and expected hitting times.