Past exam of the mathematics course of the University of Cambridge / 2018 / iii / Paper 207 / 1 / a

Write $Z(u)$ for the continuous-time Markov chain and ${\mathbb{E}}_r$ for expected value conditional on $Z(0)=r$. The occupation time of a continuous-time Markov chain in state $s$ is the integral of its indicator random variable. By Tonelli theorem,The interchange is valid because the integrand is nonnegative; indeed the random integral is bounded by $t$.

Let $N(u)$ count all jumps of the continuous-time Markov chain. Its Markov-chain entry count into state $s$ isThusThis is a Lebesgue-Stieltjes integral against a counting process, so each jump contributes one. The initial occupation of state $s$ is not counted as an entry. An ordinary time integral of an indicator random variable asserting a jump at exactly time $u$ would be zero, since individual jump times have zero Lebesgue measure. The requested integral must therefore use this counting-measure interpretation. Assume non-explosion and finite expected jump counts on bounded intervals, as holds for a finite-state continuous-time Markov chain.

For $i\ne s$, the Markov property and the definition of transition intensity giveSumming over the possible pre-jump states gives the density of the expected Markov-chain entry count:For a finite-state continuous-time Markov chain, the chance of two or more jumps in this short interval is $o(du)$. There is no $q_{ss}$ contribution: diagonal entries of the infinitesimal generator encode exit rates, not jumps into the same state. With repeated entries possible, this density is an expected event rate rather than the probability density function of a single normalized random time.

Integrate the expected Markov-chain entry count density and use the occupation time of a continuous-time Markov chain formula:Each term is a transition intensity multiplied by the expected exposure time to that transition. This counts every entry, including returns, rather than only the first visit. For countably many states the same nonnegative interchange follows from Tonelli theorem, provided the resulting expected count is finite. The sum excludes $i=s$.