= Solution
Let $N(u)$ count all jumps of the <continuous-time Markov chain>. Its <Markov-chain entry count> into state $s$ is
$$
N_s(t)=\sum_{0<u\leq t}\mathbf1\{Z(u-)\ne s,\ Z(u)=s\}
=\int_{(0,t]}\mathbf1\{Z(u)=s\}\,dN(u).
$$
Thus
$$
\boxed{E_{rs}(t)=\mathbb E_r\!\left[\int_{(0,t]}\mathbf1\{Z(u)=s\}\,dN(u)\right].}
$$
This is a <Lebesgue-Stieltjes integral> against a <counting process>, so each jump contributes one. The initial occupation of state $s$ is not counted as an entry. An ordinary time integral of an <indicator random variable> asserting a jump at exactly time $u$ would be zero, since individual jump times have zero <Lebesgue measure>. The requested integral must therefore use this counting-measure interpretation. Assume non-explosion and finite expected jump counts on bounded intervals, as holds for a finite-state <continuous-time Markov chain>.
Back to article page
