 Fundamental matrix of a finite absorbing Markov chain

For a finite Markov chain in which an absorbing state is accessible from every state, order the nonabsorbing states first and writeHere $I$ is the identity matrix and $N$ is the matrix inverse of $I-Q$. With $T$ the first hitting time of the absorbing set,Thus $N$ gives the expected occupation count before absorption, including the initial visit; $B_{ia}$ is the absorption probability of state $a$; and $t_i$ is the expected hitting time of the absorbing set. The vector $1$ has every entry one. See Grinstead and Snell, section 11.2, theorems 11.4-11.6.

For the toy routine in absorbing state, direct matrix multiplication givesThe first row of $N$ says that starting at $u$ gives $2.4$ visits to $u$ and $0.8$ to $v$ before termination. Their sum is the mean duration $3.2$. The identities $(I-Q)N=I$, $B=QB+R$, and $t=1+Qt$ independently verify the inverse, outcome, and duration calculations. The absorbing set must be reachable from every state for these finite formulas: applying them to all nonabsorbing states in the closed-cycle example would leave a persistent cycle in $Q$ and make $I-Q$ singular.