= Markov chain
{c}
{wiki}
The term Markov chain commonly denotes a <Markov process> on a finite or <countable> <state space>. Time may be discrete, as for a daily record, or continuous, as for a <continuous-time Markov chain>. Some authors also use “<Markov chain>” for discrete-time processes on more general <state spaces>; specifying both time and state space avoids this ambiguity.
For a <Discrete-time Markov chain>, its initial <probability distribution> $\mu_0$ and <transition matrix> $P$ specify the <probability> of each finite path:
$$
\mathbb P(X_0=i_0,\ldots,X_n=i_n)=\mu_0(i_0)\prod_{k=0}^{n-1}P_{i_k i_{k+1}}.
$$
This is the <Markov factorization>, obtained by applying <conditional probability> repeatedly. See https://ocw.mit.edu/courses/18-445-introduction-to-stochastic-processes-spring-2015/365d85ac35a7c006bc33dd03da633997_MIT18_445S15_lecture1.pdf[Hao Wu's MIT lecture 1, definition and distribution evolution].
For the constructed sunny/rainy <Markov chain> below, starting sunny makes the path sunny, rainy, sunny have <probability> $1\times0.2\times0.4=0.08$. Starting instead with <stationary distribution> $(2/3,1/3)$ makes that path's <probability> $(2/3)\times0.2\times0.4=4/75$. The <transition probabilities> are identical in both experiments; the initial <probability distribution> distinguishes them.
Back to article page
