Past exam of the mathematics course of the University of Cambridge / 2007 / ii / Paper 2 / 26J (Applied Probability)

For $i\ne j$ the generator entry is $q_{ij}=r_j$. Its diagonal is the negative total departure rate, $q_{ii}=-(R-r_i)$. ThusEvery row sums to zero, as required for a continuous-time Markov-chain generator.

When $R=0$ all rates vanish. Every state is absorbing and the communicating classes are the $s$ singletons, all closed. There are no open communicating classes. Every probability distribution is stationary in this degenerate case.

For $R>0$, the chain has one communicating class exactly when every $r_j>0$. Sufficiency follows from the positive direct transition rate between each distinct pair of states. For necessity, if $r_j=0$ and there is another state, no other state can jump to $j$, either directly or as the final step of a path, so $j$ cannot communicate with them. The one-state case also satisfies the condition because $R=r_1>0$. A single class containing the entire state space is automatically closed.

Let $S_{+}=\{j:r_j>0\}$, nonempty since $R>0$. Every pair of distinct states in $S_{+}$ communicates directly, and no transition leaves $S_{+}$ for a zero-rate state. Hence $S_{+}$ is the unique closed communicating class. Each zero-rate state forms an open singleton: it has total departure rate $R>0$, goes into $S_{+}$, and can never be revisited. This includes the case that $S_{+}$ itself is a single absorbing state.

The stationary equation $\pi Q=0$ reads $r^T-R\pi =0$, because $\pi 1=1$. ThereforeIt is the unique stationary law and vanishes on the open classes. For distinct $i,j$, ${\pi }_i{q}_{ij}=r_i{r}_j/R={\pi }_j{q}_{ji}$. Thus the chain is reversible in its equilibrium law, including the reducible cases; on the closed class this is the usual reversible irreducible chain. This structure is the destination-dependent reset chain.

At a state with positive departure rate ${\lambda }_i=R-r_i$, the next-jump probabilities areIf at least two rates are positive, every state has positive departure rate and the normalizing constant $Z={\sum }_i{r}_i(R-r_i)=R^2-{\sum }_i{r}_i^2$ is positive. The jump-chain equilibrium law isIndeed ${\hat{\pi }}_i{\hat{p}}_{ij}=r_i{r}_j/Z$ is symmetric, proving both stationarity and reversibility. Jump observations weight the continuous-time equilibrium by the departure rate, so $\hat{\pi }$ is generally different from $\pi$.

If exactly one rate, $r_k$, is positive, state $k$ is absorbing and the denominator $R-r_k$ vanishes. There is no next jump from that state: the genuine embedded jump sequence terminates there. Under the usual extended-chain convention set ${\hat{p}}_{kk}=1$; every other state jumps to $k$ with probability one. Its equilibrium law is ${\delta }_k$ and it is trivially reversible there. The normalized $r_i(R-r_i)$ formula cannot be used in that absorbing case.