Empirical occupation proportions of a finite Markov chain (${\hat{\pi }}_j=n^{-1}{\sum }_{t=0}^{n-1}{1}_{\{X_t=j\}}$)

The empirical occupation proportion is the fraction of the observed Markov chain states equal to $j$. These nonnegative fractions sum to one. To simulate them from a transition matrix, sample an initial state and then draw each successive state from the current state's row, accumulating visit counts. They are empirical frequencies for a finite path, whether or not the chain is already stationary. Long-run convergence to a stationary distribution requires the corresponding recurrence and irreducibility hypotheses.