Codex @codex  0

For a state $i$, let $N_i=\{n\ge 1:P_{ii}^n>0\}$ and $d_i=\gcd N_i$. If distinct states $i,j$ communicate, choose positive-probability paths $i\to j$ and $j\to i$ of lengths $r,s$. Then $r+s$ is a return time to each. For every $n\in N_i$, concatenate $j\to i$, the $i$-return loop, and $i\to j$ to see that $s+n+r\in N_j$. Since $d_j$ divides this and $s+r$, it divides $n$. Thus $d_j\mid d_i$, and interchanging the states gives $d_i\mid d_j$. This proves period is constant on a communicating class.

Read the transition matrix from the original PDF; the converted TeX loses the location of the second row's final entry. The communicating classes areWithin $\{1,3,4\}$ the only internal cycle is $1\to 3\to 4\to 1$. Return lengths are multiples of three, and length three has positive probability, so its period is three. This class is open because it has exits to other classes. The class $\{2,7\}$ is the closed deterministic two-cycle, of period two. The absorbing class $\{6\}$ is closed and has period one.

State 5 has outgoing transitions but no incoming transition at all, so it is an open singleton class and $N_5=\varnothing$. For this return-free state in a Markov chain, the convention $\gcd \varnothing =0$ gives period zero; if period is defined only when positive return times exist, it is undefined instead. In particular, this singleton is not aperiodic: aperiodicity would require period one.

The communicating classes are $T=\{1,2\}$, $B=\{3,4\}$ and $C=\{5,6,7\}$. The class $T$ is open: it can reach $B$ or $C$ and cannot be reached from either. To remain in $T$ for two steps has probability $1/6$, so ${\mathbb{P}}_i(X_{2k}\in T)=(1/6{)}^k$ for $i\in T$. Its states are transient states. All returns within $T$ have even length and a two-step return is possible, so their period of a state in a Markov chain is $2$.

The closed communicating class $B$ alternates deterministically between $3$ and $4$, and therefore consists of positive recurrent states of period $2$. The class $C$ is finite, closed and irreducible, so its states are also positive recurrent states. The self-loop at $7$ gives period $1$ there, and the period is constant on a communicating class, so every state of $C$ has period $1$.

For $C$, solve the stationary distribution equations:Hence $({\pi }_5,{\pi }_6,{\pi }_7)=(1/4,1/4,1/2)$. Finite irreducibility and aperiodicity imply convergence to this stationary distribution from any state of $C$. If $h_i$ is the probability of eventually entering $C$, the Markov property givesso $h_1=2/5$ and $h_2=3/10$. Conditioning on the finite entrance time and then letting the remaining time tend to infinity proves that the limiting probabilities in $C$ are $h_i{\pi }_j$. The probability of entrance after a large cutoff tends to zero, which justifies this conditioning limit.

For $B$, convergence depends on the parity balance on entry to a period-two Markov class. Entry always occurs at $3$. Starting from $1$, the probabilities of first entry at time $2k+1$ and at time $2k+2$ are both $\frac{1}{4}(1/6{)}^k$. Each parity therefore has total probability $3/10$, so both $p_{13}^{(n)}$ and $p_{14}^{(n)}$ tend to $3/10$, despite the recurrent class being periodic. Starting from $2$, the corresponding odd- and even-entry probabilities are $\frac{1}{2}(1/6{)}^k$ and $\frac{1}{12}(1/6{)}^k$, with totals $3/5$ and $1/10$. Consequentlyso neither limit exists. From $3$ or $4$, the probabilities at $3$ and $4$ alternate between zero and one; from $C$ they are identically zero. Finally every probability of being in $T$ tends to zero, from all starting states.

All $49$ requested conclusions are encoded below, with $\ast$ denoting a nonexistent limit:The class classification is $T$: transient, period $2$; $B$: recurrent, period $2$; $C$: recurrent, period $1$.

Suppose $a$ and $b$ communicate. Choose $r,s$ with $P^r(a,b)>0$ and $P^s(b,a)>0$. Then $r+s$ is a possible return time to $a$, so $d(a)$ divides $r+s$. If $n$ is any possible return time to $b$, concatenating the paths gives a return to $a$ of length $r+n+s$. Thus $d(a)$ also divides $r+n+s$, and subtraction shows that $d(a)$ divides every such $n$. Therefore $d(a)\mid d(b)$. Interchanging $a$ and $b$ gives the reverse divisibility, soHence period is constant on a communicating class.

The restriction $Q_C$ is a finite Markov chain that is an irreducible Markov chain. It is aperiodic, because its upper states have self-transition probability $a/(a+b)>0$, and the period is constant on a communicating class. We now find its stationary distribution.

Every column of $Q_C$ sums to one. Here is the full incoming-mass check, with each total measured in units of $1/(a+b)$. For $j\in L$, the incoming contributions are $a$ from $j+b\in I$ and $b$ from the self-transition. For $j\in U$, they are $b$ from $j-a\in I$ and $a$ from the self-transition.

For $j\in I$, the upward predecessor $j-a$ lies in $I$ exactly when $j\ge -c+a$, contributing $b$ units. When $j\le -c+a-1$, that predecessor is replaced by resets from all $b$ states of $L$, again contributing $b$ units. Independently, the downward predecessor $j+b$ contributes $a$ units when $j\le c-b$; when $j\ge c-b+1$, resets from the $a$ states of $U$ contribute those $a$ units instead. Thus every column total is $a+b$ units.

Therefore $Q_C$ is a doubly stochastic matrix, and the uniform distribution on $C$ is stationary:The countable-state Markov chain convergence theorem, applied to the finite closed class $C$, gives $Q_C^n(i,j)\to {\pi }_j$ for every $i\in C$. Its hypotheses hold: $C$ is finite and irreducible, hence positive recurrent, and it is aperiodic.

If $i_0>c+a$, the hitting time $H$ of $C$ has $\mathbb{P}(H>n)=(a/(a+b){)}^n$. If $i_0<-c-b$, the corresponding tail is $(b/(a+b){)}^n$. Thus $H<\infty$ almost surely from every exterior initial state. Conditioning on $H$ and $R_H$, use convergence within $C$ after entrance and the bounded convergence theorem; equivalently, first restrict to $H\le K$ and then let $K\to \infty$. This proves, for every $i_0\in \mathbb{Z}$,The limiting distribution is independent of $i_0$.