Codex @codex  0

Here $0<p<1$, $\theta >0$, and the upward step is either a birth or immigration, the latter having conditional probability $\theta /(k+\theta )$. At zero immigration occurs with probability one. Put $q=1-p$, $a=\theta /p$ and $r=p/q$. The stationary weights of a reflected birth-death chain areTheir successive ratio tends to $r$, so they are summable for $p<1/2$. For $p\ge 1/2$, $u_k>1/2>d_{k+1}$ makes the weights strictly increasing, ruling out a stationary distribution. Thus positive recurrence holds exactly for $p<1/2$. In that case the binomial series gives ${\sum }_k{w}_k=2(1-r{)}^{-a-1}$, andEvery step reverses parity and a two-step return has positive probability, so the period of a state in a Markov chain is two. Starting at zero, $\mathbb{P}(X_{2n}=0)\to 2{\pi }_0=(1-r{)}^{a+1}$, whereas $\mathbb{P}(X_{2n+1}=0)=0$.

For a finite, time-homogeneous Discrete-time Markov chain with row-stochastic transition matrix $P$, irreducibility meansEquivalently, all states form one communicating class. The step count may depend on $i,j$; irreducibility does not require every one-step transition probability to be positive. Drawing an arrow $i\to j$ whenever $P_{ij}>0$ makes the condition exactly that of a strongly connected directed graph. It is a property of the entire chosen state space, independent of the initial probability distribution.

Irreducibility describes where the Markov chain can travel; aperiodicity describes the possible lengths of return paths. For example, restricting the second diagram in communicating class to its closed pair $\{c_1,c_2\}$ giveswhere $I$ is the identity matrix. This Markov chain is irreducible, but each state's period of a state in a Markov chain is two. Every destination is reachable, yet a given destination cannot be reached at every sufficiently large step count. Starting at $c_1$ produces alternating one-time probability distributions. See Hao Wu's MIT lecture 2, irreducibility and periods for the definitions.

Adding a holding probability of a Markov chain $\epsilon$ at each state, with $0<\epsilon <1$, givesThe arrows between the two states remain, so the Markov chain remains irreducible. A positive one-step return makes it an aperiodic Markov chain. At $\epsilon =1/2$, both rows are $(1/2,1/2)$ and $C_{1/2}^n=C_{1/2}$ for every $n\ge 1$: the initial state is forgotten after one step. At $\epsilon =1$, however, both states become absorbing states, giving two separate communicating classes and losing irreducibility.

For a finite Markov chain, irreducibility guarantees a unique stationary distribution with positive weight at every state. Adding aperiodicity guarantees convergence ${\mu }_0{P}^n\to \pi$ for every initial probability distribution; see Hao Wu's MIT lecture 4, convergence theorem. The Markov chain ergodic theorem gives convergence of empirical visit frequencies even for an irreducible periodic Markov chain. These results explain why checking connectivity is a useful first step before studying equilibrium. Conversely, any absorbing state prevents irreducibility if there is another state: it cannot reach that state. The one-state Markov chain is the exception.

The communicating classes are $T=\{1,2\}$, $B=\{3,4\}$ and $C=\{5,6,7\}$. The class $T$ is open: it can reach $B$ or $C$ and cannot be reached from either. To remain in $T$ for two steps has probability $1/6$, so ${\mathbb{P}}_i(X_{2k}\in T)=(1/6{)}^k$ for $i\in T$. Its states are transient states. All returns within $T$ have even length and a two-step return is possible, so their period of a state in a Markov chain is $2$.

The closed communicating class $B$ alternates deterministically between $3$ and $4$, and therefore consists of positive recurrent states of period $2$. The class $C$ is finite, closed and irreducible, so its states are also positive recurrent states. The self-loop at $7$ gives period $1$ there, and the period is constant on a communicating class, so every state of $C$ has period $1$.

For $C$, solve the stationary distribution equations:Hence $({\pi }_5,{\pi }_6,{\pi }_7)=(1/4,1/4,1/2)$. Finite irreducibility and aperiodicity imply convergence to this stationary distribution from any state of $C$. If $h_i$ is the probability of eventually entering $C$, the Markov property givesso $h_1=2/5$ and $h_2=3/10$. Conditioning on the finite entrance time and then letting the remaining time tend to infinity proves that the limiting probabilities in $C$ are $h_i{\pi }_j$. The probability of entrance after a large cutoff tends to zero, which justifies this conditioning limit.

For $B$, convergence depends on the parity balance on entry to a period-two Markov class. Entry always occurs at $3$. Starting from $1$, the probabilities of first entry at time $2k+1$ and at time $2k+2$ are both $\frac{1}{4}(1/6{)}^k$. Each parity therefore has total probability $3/10$, so both $p_{13}^{(n)}$ and $p_{14}^{(n)}$ tend to $3/10$, despite the recurrent class being periodic. Starting from $2$, the corresponding odd- and even-entry probabilities are $\frac{1}{2}(1/6{)}^k$ and $\frac{1}{12}(1/6{)}^k$, with totals $3/5$ and $1/10$. Consequentlyso neither limit exists. From $3$ or $4$, the probabilities at $3$ and $4$ alternate between zero and one; from $C$ they are identically zero. Finally every probability of being in $T$ tends to zero, from all starting states.

All $49$ requested conclusions are encoded below, with $\ast$ denoting a nonexistent limit:The class classification is $T$: transient, period $2$; $B$: recurrent, period $2$; $C$: recurrent, period $1$.

The period of a state in a Markov chain $a$ isA state is aperiodic when this greatest common divisor is one.

Within the valid parameter rectangle, suppose $a>0$ and $b<1/3$. The chain is then irreducible by part (i). State $1$ has a two-step return $1\to 2\to 1$ and a three-step return $1\to 2\to 4\to 1$, both of positive probability. Its period of a state in a Markov chain therefore divides both two and three, hence equals one. Irreducibility gives the same period to every state, so the chain is aperiodic.

If $b=1/3$ and $a>0$, every transition goes between the two classes $\{1,3\}$ and $\{2,4\}$. Returns must have even length, while $1\to 2\to 1$ has length two. Thus the irreducible chain has period two.

If $a=0$, the closed communicating class $\{1,3,4\}$ alternates between $\{1,3\}$ and $\{4\}$. Its states have period two, using the positive two-step returns through $4$. Consequently the chain is not aperiodic, regardless of how the transient state $2$ is classified. Combining the cases,

After a jump from one to $k$, the first return to one occurs exactly $k$ steps later, with probability ${\alpha }_k$. Later return times are sums of such positive-support values. Thus their greatest common divisor isIn an irreducible chain this is the common period of every state. Hence the chain is aperiodic exactly when $d=1$. A self-loop at one is sufficient but not necessary; support at two and three also has greatest common divisor one.

A return path with $u$ upward steps and $v$ downward steps satisfies $au=bv$. Since $a,b$ are coprime, this is equivalent to $u=bk$ and $v=ak$ for an integer $k\ge 1$. Its length is therefore $k(a+b)$.

Conversely, for each such $k$, any ordering of $bk$ upward and $ak$ downward steps is an allowed positive-probability return path. The possible strictly positive return times are exactly the positive multiples of $a+b$. By the definition of the period of a state in a Markov chain as their greatest common divisor,In particular, none of the states is aperiodic.

Let $d=\gcd (a,b)>1$ and put $a^{\prime }=a/d$, $b^{\prime }=b/d$. Then $a^{\prime },b^{\prime }$ are coprime integers, and $p=b^{\prime }/(a^{\prime }+b^{\prime })$. The six requested revisions are as follows.

(b)(i): communicating classes. Every increment is divisible by $d$, so the residue class modulo $d$ is preserved. Conversely, if $j-i$ is divisible by $d$, Bézout's identity applied to $a^{\prime },b^{\prime }$ gives a path just as in (b)(i). Thus the original Markov chain is reducible on $\mathbb{Z}$, with exactly the $d$ closed communicating classesThe specified walk starting at zero stays in $d\mathbb{Z}$.

(b)(ii): period of a state in a Markov chain. The return equation $au=bv$ reduces to $a^{\prime }u=b^{\prime }v$. Hence $u=b^{\prime }k$, $v=a^{\prime }k$. For every state, the period of a state in a Markov chain is thereforeThis is still at least two, so no state is aperiodic.

(b)(iii): stationary distribution. The conclusion is unchanged: there is no stationary distribution on $\mathbb{Z}$. The maximum-mass proof in (b)(iii) applies verbatim, without using coprimality. Equivalently, each closed communicating class, after translating and dividing its state labels by $d$, is the same centered random walk with step sizes $a^{\prime },-b^{\prime }$ and has no stationary distribution.

(b)(iv): occupation of zero. On $d\mathbb{Z}$, the rescaled walk $S_n/d$ has exactly the transition probabilities of the coprime walk with parameters $a^{\prime },b^{\prime }$. The return-probability argument in (b)(iv) therefore makes this class null recurrent. The renewal process proof of the occupation limit applies unchanged:Starting in another residue class would make every visit to zero impossible, also giving occupation proportion zero.

(c)(i): modified communicating classes. These are unchanged: $C=\{-c-b,\dots ,c+a\}$ is one finite closed communicating class, and all other states are transient singleton classes. The original proof used consecutive reset destinations and never used coprimality. Resetting to the $b$ consecutive integers $c-b+1,\dots ,c$ connects different residue classes, so it would be incorrect to split $C$ according to residue classes modulo $d$.

(c)(ii): limiting distribution. The column-sum calculation, the irreducible Markov chain property on a finite class, positive self-transition probabilities, and geometric entrance argument also required no coprimality. Thus for every initial state the same convergence in distribution holds:In particular, the denominator $2c+a+b+1$ is unchanged; it is not divided by $d$.

Every step changes the population by one, so $X_n$ has the parity of $n$ when $X_0=0$. Hence $\mathbb{P}(X_{2m+1}=0)=0$. A return in two steps has positive probability from every state, and all return times are even, so the period of a state in a Markov chain is exactly two.

Each transition exchanges the even and odd classes. Since $\pi$ is an invariant distribution, their masses are equal: $\pi (\text{even})=\pi (\text{odd})=1/2$. The chain sampled at times $2m$, restricted to the even states, is irreducible and positive recurrent, and it is aperiodic because its diagonal entries are positive. Its stationary distribution is $2{\pi }_k$ on even $k$. The countable-state Markov chain convergence theorem applied to this sampled chain yieldsSince ${\pi }_0>0$, the full sequence has no limit. Its time average nevertheless tends to ${\pi }_0$ by the countable-state Markov chain ergodic theorem.

A stationary distribution is a probability distribution $\pi$ preserved by a Markov chain's evolution. For a Discrete-time Markov chain in the row convention,Thus $\pi$ is a normalized nonnegative left eigenvector of the transition matrix for eigenvalue one. Starting with $X_0$ distributed as $\pi$ makes every $X_n$ have that same probability distribution. For a time-homogeneous Markov chain, the path probabilities are also unchanged by shifting all observation times together: the process has strict stationarity. This does not mean its individual paths are constant. See Hao Wu's MIT lecture 2, stationary distributions.

For the sunny/rainy transition matrix, write $\pi =(a,1-a)$. The sunny component of $\pi P=\pi$ gives $a=0.8a+0.4(1-a)$, so $0.6a=0.4$ andMoreover,These equal stationary flows verify detailed balance and show that the Markov chain is reversible. The diagonal detailed balance equations hold automatically. Only $1/3$ of the stationary days are rainy, but $0.6$ of the days following a rainy day are rainy: stationarity permits dependence.

A finite Markov chain that is an irreducible Markov chain has a unique stationary distribution. If it is also an aperiodic Markov chain, ${\mu }_0{P}^n\to \pi$ for every initial probability distribution. The sunny/rainy example satisfies both conditions: both directions are possible and each state has a positive holding probability of a Markov chain. See Hao Wu's MIT lecture 4, convergence theorem. For an infinite state space, the additional positive recurrence hypothesis in the countable-state Markov chain convergence theorem matters.

Existence of a stationary distribution alone does not give this convergence. Consider the deterministic two-cycleHere $\pi C=\pi$, but starting in the first state gives alternating probability distributions $(1,0),(0,1),(1,0),\dots$. Its period of a state in a Markov chain is two. The running fraction of visits to either state nevertheless tends to $1/2$, illustrating the Markov chain ergodic theorem. Starting with $\pi$ instead keeps the one-time probability distributions stationary even though every path alternates.

Take $d$ independent copies of the simple random walk on the integer line, each stepping left or right with equal probability at every integer time. Their product Markov chain moves all coordinates at once; each of its $2^d$ diagonal increments has probability $2^{-d}$. At odd times a return to the origin is impossible. At time $2n$, independence gives the displayed return probability, whose asymptotic equivalence follows from Stirling's approximation. The recurrence criterion by return probabilities and the p-series therefore imply recurrence for $d=1,2$ and a transient Markov chain for $d\ge 3$.

For $d>1$ the chain from the origin visits only the sublattice whose coordinates all have the same parity. This is one communicating class; restricting to it gives an irreducible Markov chain. The period of a state in a Markov chain is two here. The conclusion applies to that class, without incorrectly claiming Markov chain irreducibility on all of ${\mathbb{Z}}^d$. This walk moves diagonally; the usual axis-step simple random walk has different transition probabilities, even though the same dimensional recurrence threshold holds.