Past exam of the mathematics course of the University of Cambridge / 2019 / ii / Paper 3 / 27K (Applied Probability)

A continuous-time Markov chain is a reversible Markov chain in equilibrium if, when started in a stationary distribution $\pi$, the process $(X_t:0\le t\le T)$ has the same finite-dimensional distributions as $(X_{T-t}:0\le t\le T)$ for every $T>0$.

For transition rates $q_{ij}$, the detailed balance for a continuous-time Markov chain equations areThey imply invariance directly. For each state $j$,because $q_{jj}=-{\sum }_{k\ne j}{q}_{jk}$. Thus $\pi Q=0$, so $\pi$ is an invariant distribution of a continuous-time Markov chain.

Put $K=N+s$, the total capacity. The customer count is an M-M-s-K queue, hence a birth-death process on $\{0,1,\dots ,K\}$. Its nonzero transition rates areThe first rate is zero at $K$ because an arrival finding the shop full is rejected. The second rate is the number of busy servers times the individual service rate.

Write $\rho =\lambda /\mu$. The detailed balance for a birth-death process recursion isThereforewhereThis is the stationary distribution of an M-M-s-K queue.

The state space is finite, and every state communicates with every other through successive births and deaths because $\lambda ,\mu >0$. The chain is therefore an irreducible Markov chain, so its invariant distribution is unique. Finally the adjacent-state identitiesare detailed balance, while all nonadjacent rates vanish in both directions. Part a now shows that the queue is reversible in equilibrium.