Markov additive process (source code)

= Markov additive process
{c}
{wiki}

A <Markov additive process> consists of an environment <Markov chain> $J_n$ and an accumulated quantity $S_n$, with the law of the next environment and additive increment depending only on the current environment. In discrete time, a family of kernels $K_{ij}$ on the <real line> specifies
$$
\mathbb P(J_{n+1}=j,\ S_{n+1}-S_n\in B\mid\mathcal F_n)=K_{J_n,j}(B),\qquad \sum_jK_{ij}(\mathbb R)=1.
$$
Here $\mathcal F_n$ contains the history through time $n$. The environment has <transition matrix> $P_{ij}=K_{ij}(\mathbb R)$. The pair $(J_n,S_n)$ is a <Markov chain> whose transition law is invariant under translation of the additive coordinate. The increments of $S$ need not be <independent random variables>[independent]: their dependence is carried by $J$. In general $S$ alone is not a <Markov chain>.

For example, an environment with two states can persistently switch between favorable and unfavorable conditions. Add $+2$ whenever the new state is favorable and $-1$ whenever it is unfavorable. Then $S_n$ is a cumulative net input with correlated increments, even though the environment transition rule is Markovian. Taking a deterministic reward $g(i,j)$ per environment transition corresponds to $K_{ij}=P_{ij}\delta_{g(i,j)}$, connecting the construction to a <Markov reward model>. More general kernels allow a random reward conditional on each transition. The construction models accumulated claims, arrivals or net gains whose statistics change with a finite-state environment.