= Discrete-time Markov chain martingale characterization
{title2=$M_n^f=f(X_n)-f(X_0)-\sum_{j<n}(Pf-f)(X_j)$}
An adapted process on a finite or countable <state space> has the <Markov chain> <transition matrix> $P$ exactly when the displayed process is a <martingale> for every bounded test <function> $f$. The forward implication follows by taking the <conditional expectation> of its increment. Conversely, the <Martingale> property gives $\mathbb E[f(X_{n+1})\mid\mathcal F_n]=(Pf)(X_n)$; testing individual-state <indicator functions>[indicators] recovers the entire conditional transition law. Stopping these <martingales> turns equations $(P-I)h=0$ and $(P-I)v=-1$ into hitting <probabilities> and expected <hitting times>.
Back to article page
