Martingale problem for a continuous-time Markov chain (source code)

= Martingale problem for a continuous-time Markov chain

Let $X$ be an <adapted process>[adapted] <Càdlàg process> on a <state space>[finite state space], and let $Q$ be a conservative <transition-rate matrix>. It solves the <martingale> problem for $Q$ when, for every real <function> $f$ on the <state space>,
$$
M_t^f=f(X_t)-f(X_0)-\int_0^t(Qf)(X_s)\,ds
$$
is a <martingale> for the specified <filtration (probability theory)>[filtration]. On a <state space>[finite state space] all such <test function>[test functions] and their generator values are bounded.

A <Continuous-time Markov chain> with generator $Q$ has this property by <Dynkin formula>[Dynkin's formula]. Conversely, fix a terminal time $t$ and put $u(s,x)=(e^{(t-s)Q}f)(x)$ for $0\leq s\leq t$. The <Kolmogorov backward equation>[backward equation] gives $\partial_su+Qu=0$. Applying the <martingale> identities to its finitely many time-dependent coefficients shows $u(s,X_s)$ is a <martingale>. Thus
$$
\mathbb E[f(X_t)\mid\mathcal F_s]=(e^{(t-s)Q}f)(X_s).
$$
Testing state indicators recovers the entire conditional transition law, proving the <Markov property> and identifying its semigroup. The <càdlàg function>[càdlàg] and <filtration (probability theory)>[filtration] assumptions make this a path-process characterization, rather than a statement only about separately fixed-time versions.