= Solution
The only <closed communicating class> is $\{b,d\}$; the class $\{a,c\}$ is open and transient. Its one-step probability of staying inside is at most $2/3$, so if $\tau$ is the entrance time into $\{b,d\}$, then $\mathbb P(\tau>n)\le(2/3)^n$ for an initial state in $\{a,c\}$. Hence entrance occurs almost surely in finite time.
On $\{b,d\}$ it is an irreducible <aperiodic Markov chain>, because both states have positive self-loop probabilities. The finite irreducible aperiodic <countable-state Markov chain convergence theorem> says that its transition probabilities converge to its unique <stationary distribution>. The balance equation $\pi_b(3/4)=\pi_d(2/3)$ with $\pi_b+\pi_d=1$ gives $\pi_b=8/17$ and $\pi_d=9/17$.
This limit also holds from $a,c$: condition on the finite entrance time and entrance state. For $\tau\le M$, convergence in the closed class applies to a finite sum; the remaining probability is bounded by $\mathbb P(\tau>M)$, which tends to zero as $M\to\infty$. Thus \b[in the order $a,b,c,d$],
$$
\boxed{\lim_{n\to\infty}P^n=
\begin{pmatrix}
0&8/17&0&9/17\\
0&8/17&0&9/17\\
0&8/17&0&9/17\\
0&8/17&0&9/17
\end{pmatrix}.}
$$
Back to article page
