= Discrete-time Markov chain martingale characterization
{title2=$M_n^f=f(X_n)-f(X_0)-\sum_{j<n}(Pf-f)(X_j)$}
An adapted process on a finite or countable state space has the <Markov chain> transition matrix $P$ exactly when the displayed process is a <martingale> for every bounded test function $f$. The forward implication follows by taking the <conditional expectation> of its increment. Conversely, the martingale property gives $\mathbb E[f(X_{n+1})\mid\mathcal F_n]=(Pf)(X_n)$; testing individual-state indicators recovers the entire conditional transition law. Stopping these martingales turns equations $(P-I)h=0$ and $(P-I)v=-1$ into hitting probabilities and expected hitting times.
Back to article page
