Past exam of the mathematics course of the University of Cambridge / 2010 / ii / Paper 1 / 27I (Applied Probability)

An equilibrium distribution of a continuous-time Markov chain is a probability row vector $\pi$ invariant under the transition semigroup: $\pi P(t)=\pi$ for every $t\ge 0$. Differentiating gives $\pi Q=0$, or ${\sum }_i{\pi }_i{q}_{ij}=0$ for each $j$, when the generator sums are defined. Reducible chains may have several closed classes supporting different equilibrium distributions; convex mixtures are then also equilibria. Irreducibility and positive recurrence give uniqueness for a conservative nonexplosive chain.

For an irreducible, nonexplosive, positive recurrent continuous-time Markov chain on a countable state space, there is a unique equilibrium distribution $\pi$ andIn particular every finite irreducible chain has this convergence. Continuous time removes the discrete-time periodicity obstruction. Reducible finite chains instead converge according to absorption probabilities into their closed classes.

The stationary equations simplify by setting $a_i={\pi }_i{\lambda }_i$. They read $2a_i={\sum }_{j\ne i}{a}_j$, so all three $a_i$ are equal. If all rates are positive, this gives the unique equilibriumIf any rate is zero, the common $a_i$ is zero, forcing zero equilibrium mass on every positive-rate state. Every zero-rate state is absorbing. With one such state $z$, the unique equilibrium is ${\delta }_z$; with two, all probability mixtures on those two states are equilibria, giving nonuniqueness. The excluded all-zero case would make every distribution an equilibrium.

When all rates are positive, finite irreducible convergence gives $P(t)\to 1\pi$, with $\pi$ as above. With one zero-rate state $z$, each visit to either transient state has probability $1/2$ of entering $z$ on the next jump. Survival for $n$ jumps therefore has probability $2^{-n}$, so absorption is almost sure andWith two zero-rate states $z_1,z_2$ and the sole positive-rate state $k$, absorption occurs at the first jump, with probability $1/2$ into each absorbing state. The limit rows are $e_{z_1}^T,e_{z_2}^T$ for those states and $(e_{z_1}^T+e_{z_2}^T)/2$ for row $k$. These limits depend on the initial state, as nonuniqueness requires.

At a positive-rate state the jump chain moves to either other state with probability $1/2$, independently of the holding rate. At a zero-rate state there is no subsequent jump; extend the chain as permanently absorbed there, giving an identity row. This convention is needed to speak of all powers of its transition matrix in the absorbing cases.

When all rates are positive,Its unique equilibrium is uniform and its powers tend to the matrix with all rows $(1/3,1/3,1/3)$. This differs from the holding-time-weighted equilibrium of the continuous-time chain unless the rates coincide.

With one zero-rate state the unique jump-chain equilibrium is its point mass and the limit has all rows that point mass. With two zero-rate states the equilibria are all mixtures on them, and the limiting rows are exactly the absorbing rows and half-half transient row found in part (d).