A finite Markov chain is a Markov chain with a finite state space, represented in discrete time by a transition matrix whose entries are nonnegative and whose rows sum to one. A row-vector distribution evolves by . Every such chain has a stationary distribution; irreducibility makes it unique and positive, while aperiodicity additionally ensures convergence from every starting distribution. For the stationary distribution is , but a chain started at one state alternates forever and does not converge.[ref]
New to topics? Read the docs here!