Solution (source code)

= Solution

The <diagonalization> in part (b) gives $M^k=P\operatorname{diag}(a^k,b^k,c^k)P^{-1}$. Hence
$$
\boxed{M^k\to0\ \Longleftrightarrow\ |a|<1,\ |b|<1,\ |c|<1.}
$$
Sufficiency follows from the scalar powers tending to zero; necessity also follows immediately from the diagonal entries of $M^k$. Under these conditions the <matrix geometric series> converges, and its sum is
$$
(I-M)^{-1}=P\operatorname{diag}\left(\frac1{1-a},\frac1{1-b},\frac1{1-c}\right)P^{-1}.
$$
Writing $A_0=1/(1-a)$, $B_0=1/(1-b)$ and $C_0=1/(1-c)$, all entries are explicitly
$$
\boxed{(I-M)^{-1}=\begin{pmatrix}A_0&B_0-A_0&C_0-B_0\\0&B_0&C_0-B_0\\0&0&C_0\end{pmatrix}.}
$$
For example, $B_0-A_0=(b-a)/[(1-a)(1-b)]$ and $C_0-B_0=(c-b)/[(1-b)(1-c)]$. These expressions remain valid when parameter values coincide.