Past exam of the mathematics course of the University of Cambridge / 2008 / ii / Paper 3 / 25I (Applied Probability)

An irreducible continuous-time Markov chain is positive recurrent when the mean return time to a state is finite, and null recurrent when return occurs almost surely but the mean return time is infinite. For the nonexplosive chain here, whose rates are bounded, positive recurrence is equivalent to existence of a normalized stationary distribution. All rates are understood to be positive.

The stationary balance equations at level zero are $(\lambda +\alpha ){\pi }_{0C}=\beta {\pi }_{0W}$ and $\beta {\pi }_{0W}=\alpha {\pi }_{0C}+\mu {\pi }_{1W}$. At every $i\ge 1$ they areThe zero-level equations give ${\pi }_{0C}=\beta {\pi }_{0W}/(\lambda +\alpha )$ and ${\pi }_{1W}=\beta \lambda {\pi }_{0W}/[\mu (\lambda +\alpha )]$. Substituting the latter in the level-one $C$ equation givesSolve the $W$ equation for ${\pi }_{i+1,W}$, then substitute it into the next $C$ equation. This yields the required row-vector recurrence with

For this alternating arrival and service queue, set $\theta =\lambda (\mu +\beta )/[\mu (\lambda +\alpha )]$. Direct multiplication gives $(\theta ,\lambda /\mu )B=\theta (\theta ,\lambda /\mu )$, so the initial row is a left eigenvector and $({\pi }_{iC},{\pi }_{iW})={\theta }^{i-1}({\pi }_{1C},{\pi }_{1W})$. ConsequentlyThese nonnegative probabilities can be normalized exactly when $\theta <1$, equivalently $\mu \alpha >\lambda \beta$. Summing the geometric series givesWhen $\theta \ge 1$ no positive multiple of the invariant sequence is summable, so no stationary distribution exists. This rules out positive recurrence; absence of a stationary probability alone should not be used to distinguish null recurrence from transience.