= Fundamental matrix of a finite absorbing Markov chain
{c}
{title2=$N=(I-Q)^{-1}$}
For a <finite Markov chain> in which an <absorbing state> is accessible from every state, order the nonabsorbing states first and write
$$
P=\begin{pmatrix}Q&R\\0&I\end{pmatrix},\qquad
N=(I-Q)^{-1}=\sum_{n\geq0}Q^n.
$$
Here $I$ is the <identity matrix> and $N$ is the <matrix inverse> of $I-Q$. With $T$ the first <hitting time> of the absorbing set,
$$
N_{ij}=\mathbb E_i\sum_{n=0}^{T-1}\mathbf1_{\{X_n=j\}},\qquad
B=NR,\qquad t=N\mathbf1.
$$
Thus $N$ gives the <expected occupation count before absorption>, including the initial visit; $B_{ia}$ is the <absorption probability> of state $a$; and $t_i$ is the <expected hitting time> of the absorbing set. The <vector> $\mathbf1$ has every entry one. See https://math.dartmouth.edu/~prob/prob/prob.pdf#page=425[Grinstead and Snell, section 11.2, theorems 11.4-11.6].
For the toy routine in <absorbing state>, direct <matrix multiplication> gives
$$
N=\begin{pmatrix}12/5&4/5\\4/5&8/5\end{pmatrix},\qquad
B=\begin{pmatrix}3/5&2/5\\1/5&4/5\end{pmatrix},\qquad
t=\begin{pmatrix}16/5\\12/5\end{pmatrix}.
$$
The first row of $N$ says that starting at $u$ gives $2.4$ visits to $u$ and $0.8$ to $v$ before termination. Their sum is the mean duration $3.2$. The identities $(I-Q)N=I$, $B=QB+R$, and $t=\mathbf1+Qt$ independently verify the inverse, outcome, and duration calculations. The absorbing set must be reachable from every state for these finite formulas: applying them to all nonabsorbing states in the closed-cycle example would leave a persistent cycle in $Q$ and make $I-Q$ singular.
Back to article page
