Transition-completion time in a Markov chain
ID: transition-completion-time-in-a-markov-chain
For a Markov chain with transition matrix , fix a directed transition with . Its transition-completion time isIt counts the successful final step. Merely visiting by another route does not complete this event. In a finite irreducible chain its mean is finite and first-step analysis givesThe subtracted continuation term removes exactly the case where the designated transition has just succeeded. An equivalent construction sends that transition to a new absorbing success state and computes its Expected hitting time.
For the two-state chain , , with , the waiting time for has means and . Starting at one, the hitting time of state one is zero, whereas the transition-completion time must first wait for and then . This example makes the difference between an edge event and a state event explicit.
New to topics? Read the docs here!