= Markov process
{c}
{wiki}
A Markov process is a <stochastic process> whose present state contains all the information in its observed past needed to predict its future. Precisely, if $\mathcal F_s$ is the <natural filtration> generated by observations up to time $s$, then, for $t>s$ and any <measurable set> $A$ of states,
$$
\mathbb P(X_t\in A\mid\mathcal F_s)=\mathbb P(X_t\in A\mid X_s).
$$
The equality is between <conditional probabilities> and holds <almost surely>. It expresses the <Markov property>: the past and future have <conditional independence> given the present. It does not assert <independence> of consecutive states. In the sunny/rainy example below, tomorrow's sunny <probability> is $0.8$ after sunshine but $0.4$ after rain, so knowing today's state matters.
The memory assumption concerns the chosen <state space>. If an equipment model's failure <probability> depends on its age, recording only “working” or “failed” may lose relevant information; recording age as part of the state can retain it. Similarly, a dependence on two previous observations can sometimes be represented by using their ordered pair as the current state. Whether such a description fits observations is a modeling question.
Transition <Markov kernels> describe the <conditional distribution> of a state at a later time. Successive kernels and the initial <probability distribution> determine every <finite-dimensional distribution>, through repeated <conditional probability>. The <time-homogeneous Markov property> is an additional assumption: the kernel depends on elapsed time rather than calendar time. The attraction is compression: prediction can proceed from a current state instead of an entire recorded history. <Brownian motion> and <continuous-time Markov chains> illustrate that a <Markov process> need not evolve in discrete time.
Back to article page
