 Markov additive process

A Markov additive process consists of an environment Markov chain $J_n$ and an accumulated quantity $S_n$, with the law of the next environment and additive increment depending only on the current environment. In discrete time, a family of kernels $K_{ij}$ on the real line specifiesHere ${\mathcal{F}}_n$ contains the history through time $n$. The environment has transition matrix $P_{ij}=K_{ij}(\mathbb{R})$. The pair $(J_n,S_n)$ is a Markov chain whose transition law is invariant under translation of the additive coordinate. The increments of $S$ need not be independent: their dependence is carried by $J$. In general $S$ alone is not a Markov chain.

For example, an environment with two states can persistently switch between favorable and unfavorable conditions. Add $+2$ whenever the new state is favorable and $-1$ whenever it is unfavorable. Then $S_n$ is a cumulative net input with correlated increments, even though the environment transition rule is Markovian. Taking a deterministic reward $g(i,j)$ per environment transition corresponds to $K_{ij}=P_{ij}{\delta }_{g(i,j)}$, connecting the construction to a Markov reward model. More general kernels allow a random reward conditional on each transition. The construction models accumulated claims, arrivals or net gains whose statistics change with a finite-state environment.