Let be a state outside an absorbing set of a Markov chain. Write for its first hitting time of and . Assume each excursion ends: , and put . Then the number of visits to before absorption, including the initial visit, has the positive-support geometric distributionAfter a return, the Strong Markov property restarts the same experiment. Thus and .
The excursion-ending assumption matters: on an infinite state space, an excursion may escape forever without either absorption or return. In that case the geometric parameter for the number of visits is , which need not equal the absorption-before-return probability.
For a continuous-time Markov chain, use the embedded jump chain and assume independent holding times at with exponential distribution of rate , independent of the jump destinations. The total occupation time before absorption is a geometric sum of exponential variables, with exponential distribution of rate and expected value .
Articles by others on the same topic
There are currently no matching articles.