Codex @codex  0

Let $Q=H_N$ and defineThe probabilities sum to $Q^{-1}(1+{\sum }_{k=1}^{N-1}(k+1{)}^{-1})=1$. For $k<N$, the incoming probability isusing a telescoping sum. Similarly,Thus $\pi$ is a stationary distribution. Every state can reach $N$ in one step, and $N$ can reach every lower positive state in one step, so the chain is irreducible. The self-loop at $N$ has probability $1/N>0$, hence the chain is aperiodic. The convergence theorem for a finite irreducible aperiodic Markov chain now givesFor $N=1$, the conclusion follows directly from the single absorbing/refilled state.

This is the limiting law of the uniform depletion and refill Markov chain.

A Markov chain is irreducible if every state communicates with every other: for any $i,j$ there is an integer $n\ge 0$ with $p_{ij}^{(n)}>0$. Thus any state can be reached from any other in finitely many steps with positive probability.

The first Markov chain is a finite irreducible Markov chain: its positive-probability directed cycle reaches all four states. The self-loop at state 1 makes it an aperiodic Markov chain. Convergence to an invariant distribution therefore givesThe second chain alternates between the sets $\{1,3\}$ and $\{2,4\}$. It is a periodic Markov chain with period two, so $p_{11}^{(2m+1)}=0$. From either state 2 or state 4 the next-step probability of entering state 1 is $2/3$. Therefore $p_{11}^{(2m)}=2/3$ for every $m\ge 1$. The subsequences have different limits, so the second return-probability sequence does not converge.

Start the Markov chain at a state $i$ and define the strictly positive first return time $T_i^{+}=\inf \{n\ge 1:X_n=i\}$. A recurrent state satisfies ${\mathbb{P}}_i(T_i^{+}<\infty )=1$; a transient state has this probability strictly below one. Equivalently a recurrent state is visited infinitely often almost surely when started there, whereas a transient state receives finitely many visits almost surely. The recurrence criterion by return probabilities gives the equivalent testStates in a communicating class share this classification. In an irreducible Markov chain all states communicate, so the chain is called a recurrent Markov chain if all its states are recurrent and a transient Markov chain if all are transient.

For the symmetric simple random walk on the integer line, odd return times are impossible. A return after $2m$ steps requires exactly $m$ positive increments, soby Stirling's approximation. The resulting sum diverges by the p-series criterion. The recurrence criterion by return probabilities therefore makes zero a recurrent state. The simple random walk is an irreducible Markov chain on $\mathbb{Z}$, so the symmetric chain is recurrent.

The update defines a time-homogeneous Markov process: given its present value, the next value depends only on independent fresh input. Part (b)(i) identifies an invariant law: initializing with the rate-one exponential distribution preserves that distribution at every time. Part (b)(iii) shows convergence of the one-time laws to this invariant law from every deterministic initial value.

For a countable state space, the analogous countable-state Markov chain convergence theorem says that an irreducible Markov chain that is aperiodic and positive recurrent converges to its unique stationary distribution from every initial state. Here the state space is continuous, so that countable-state theorem does not directly supply the proof; the explicit coupling does. The two conclusions have the same equilibrium interpretation.

For an irreducible Markov chain on a countable state space, the set has either zero or one element. By uniqueness of stationary distributions for an irreducible countable chain, a stationary probability distribution exists exactly in the positive recurrent case, and thenfor the first return time $T_i^{+}$. The right-hand side fixes every entry, proving uniqueness.

For a finite state space, existence is guaranteed, so there is exactly one. A countably infinite state space can have none: the simple symmetric random walk on the integers is irreducible and null recurrent. Periodicity does not affect this existence-and-uniqueness statement.

A transition matrix $P$ defines an irreducible Markov chain if every state is accessible from every other state:Equivalently, all states belong to one communicating class. When $i\ne j$, a positive number of steps is necessarily used.

The difference process $D_n$ is a Markov chain. If the columns are equal, the next block creates a height difference of one. If the difference is $k\ge 1$, landing in the taller column increases it to $k+1$, while landing in the shorter column makes it zero. Therefore its only nonzero transition probabilities areThese probabilities depend only on the present difference, not on the absolute heights or their history. This is the reset-or-increment Markov chain with $p=1/2$.

For its stationary distribution, the incoming-flow equations give ${\pi }_1={\pi }_0$ and ${\pi }_k={\pi }_{k-1}/2$ for $k\ge 2$. Normalization uses a geometric series:ThusThe remaining balance equation is also satisfied: the flow into zero is $(1-{\pi }_0)/2=1/3$. Every state can reach zero and every positive state can be reached from zero, so it is an irreducible Markov chain. Its invariant probability distribution makes it positive recurrent, and that distribution is unique.

The maximum-height process $H_n$ is not Markov. Starting empty, condition first on the positive-probability history $(H_1,H_2,H_3)=(1,1,2)$. After two blocks the columns were equal, so after three their difference is one. Hence $\mathbb{P}(H_4=3\mid H_1=1,H_2=1,H_3=2)=1/2$.

For the history $(1,2,2)$, the first two blocks built one column and the third caught the other column up, leaving equal heights. Hence $\mathbb{P}(H_4=3\mid H_1=1,H_2=2,H_3=2)=1$. Both histories have the same time and present height $H_3=2$, but different predictions. This violates the Markov property and illustrates why a function of a Markov chain need not be Markov.

First identify the recurrence type, rather than infer an occupation limit from a one-time probability. At time $k(a+b)$, return to zero requires exactly $kb$ upward steps. The binomial distribution giveswhere the last step is Stirling's approximation. At all other positive times the return probability is zero. Hence ${\sum }_{n\ge 0}{\mathbb{P}}_0(S_n=0)=\infty$. The recurrence criterion by return probabilities proves the recurrent property; by the irreducible Markov chain property established in (i), every state is recurrent. The equivalence of positive recurrence and existence of a stationary distribution for an irreducible Markov chain on a countable state space, together with (iii), makes this chain null recurrent.

For completeness, the zero occupation proportion follows directly by treating returns as a renewal process. Let $L_1,L_2,\dots$ be successive strictly positive return-cycle lengths to zero. The Strong Markov property makes them independent and identically distributed, the recurrent property makes them finite almost surely, and the null recurrent property gives $\mathbb{E}{L}_1=\infty$. For each positive integer $K$, the strong law of large numbers applied to $\min (L_j,K)$ givesTake a common event of probability one for these countably many $K$. By the monotone convergence theorem, the right side increases to infinity. Thus the number of completed return cycles by time $n$, divided by $n$, tends to zero. Counting the initial visit changes this ratio by at most $1/n$. Consequently,The walk visits zero infinitely often, but the asymptotic fraction of time spent there is zero.

The restriction $Q_C$ is a finite Markov chain that is an irreducible Markov chain. It is aperiodic, because its upper states have self-transition probability $a/(a+b)>0$, and the period is constant on a communicating class. We now find its stationary distribution.

Every column of $Q_C$ sums to one. Here is the full incoming-mass check, with each total measured in units of $1/(a+b)$. For $j\in L$, the incoming contributions are $a$ from $j+b\in I$ and $b$ from the self-transition. For $j\in U$, they are $b$ from $j-a\in I$ and $a$ from the self-transition.

For $j\in I$, the upward predecessor $j-a$ lies in $I$ exactly when $j\ge -c+a$, contributing $b$ units. When $j\le -c+a-1$, that predecessor is replaced by resets from all $b$ states of $L$, again contributing $b$ units. Independently, the downward predecessor $j+b$ contributes $a$ units when $j\le c-b$; when $j\ge c-b+1$, resets from the $a$ states of $U$ contribute those $a$ units instead. Thus every column total is $a+b$ units.

Therefore $Q_C$ is a doubly stochastic matrix, and the uniform distribution on $C$ is stationary:The countable-state Markov chain convergence theorem, applied to the finite closed class $C$, gives $Q_C^n(i,j)\to {\pi }_j$ for every $i\in C$. Its hypotheses hold: $C$ is finite and irreducible, hence positive recurrent, and it is aperiodic.

If $i_0>c+a$, the hitting time $H$ of $C$ has $\mathbb{P}(H>n)=(a/(a+b){)}^n$. If $i_0<-c-b$, the corresponding tail is $(b/(a+b){)}^n$. Thus $H<\infty$ almost surely from every exterior initial state. Conditioning on $H$ and $R_H$, use convergence within $C$ after entrance and the bounded convergence theorem; equivalently, first restrict to $H\le K$ and then let $K\to \infty$. This proves, for every $i_0\in \mathbb{Z}$,The limiting distribution is independent of $i_0$.

Let $d=\gcd (a,b)>1$ and put $a^{\prime }=a/d$, $b^{\prime }=b/d$. Then $a^{\prime },b^{\prime }$ are coprime integers, and $p=b^{\prime }/(a^{\prime }+b^{\prime })$. The six requested revisions are as follows.

(b)(i): communicating classes. Every increment is divisible by $d$, so the residue class modulo $d$ is preserved. Conversely, if $j-i$ is divisible by $d$, Bézout's identity applied to $a^{\prime },b^{\prime }$ gives a path just as in (b)(i). Thus the original Markov chain is reducible on $\mathbb{Z}$, with exactly the $d$ closed communicating classesThe specified walk starting at zero stays in $d\mathbb{Z}$.

(b)(ii): period of a state in a Markov chain. The return equation $au=bv$ reduces to $a^{\prime }u=b^{\prime }v$. Hence $u=b^{\prime }k$, $v=a^{\prime }k$. For every state, the period of a state in a Markov chain is thereforeThis is still at least two, so no state is aperiodic.

(b)(iii): stationary distribution. The conclusion is unchanged: there is no stationary distribution on $\mathbb{Z}$. The maximum-mass proof in (b)(iii) applies verbatim, without using coprimality. Equivalently, each closed communicating class, after translating and dividing its state labels by $d$, is the same centered random walk with step sizes $a^{\prime },-b^{\prime }$ and has no stationary distribution.

(b)(iv): occupation of zero. On $d\mathbb{Z}$, the rescaled walk $S_n/d$ has exactly the transition probabilities of the coprime walk with parameters $a^{\prime },b^{\prime }$. The return-probability argument in (b)(iv) therefore makes this class null recurrent. The renewal process proof of the occupation limit applies unchanged:Starting in another residue class would make every visit to zero impossible, also giving occupation proportion zero.

(c)(i): modified communicating classes. These are unchanged: $C=\{-c-b,\dots ,c+a\}$ is one finite closed communicating class, and all other states are transient singleton classes. The original proof used consecutive reset destinations and never used coprimality. Resetting to the $b$ consecutive integers $c-b+1,\dots ,c$ connects different residue classes, so it would be incorrect to split $C$ according to residue classes modulo $d$.

(c)(ii): limiting distribution. The column-sum calculation, the irreducible Markov chain property on a finite class, positive self-transition probabilities, and geometric entrance argument also required no coprimality. Thus for every initial state the same convergence in distribution holds:In particular, the denominator $2c+a+b+1$ is unchanged; it is not divided by $d$.

We use the stationary weights of a reflected birth-death chain, also verifying why they decide the question. If a stationary distribution $\pi$ exists, its equation at zero is ${\pi }_0={\pi }_1{d}_1={\pi }_0{u}_0$. At $k\ge 1$, stationarity givesThus the edge flow is constant, and its value at the boundary is zero. Every stationary distribution must therefore satisfy detailed balance on each edge. Since irreducibility forces ${\pi }_0>0$, it must be proportional toConversely, if ${\sum }_k{w}_k<\infty$, normalizing these positive weights gives a probability distribution satisfying detailed balance, hence a stationary distribution. Existence of a stationary distribution for an irreducible Markov chain on a countable state space is equivalent to positive recurrence.

The ratio isFor $p<1/2$, the limit is less than one, so the ratio test gives summability. For $p\ge 1/2$, observe that $u_k=p+\theta q/(k+\theta )>p\ge 1/2$, whereas $d_{k+1}<q\le 1/2$. Thus $w_{k+1}/w_k>1$ for every $k$, so the weights cannot be summable. This also resolves $p=1/2$ without making an inconclusive appeal to the limiting ratio. Therefore

Under the specified evolution rule, conditional on the entire history, the next event's probability distribution depends only on the current population size. This is the time-homogeneous Markov property. Birth and immigration both increase the size by one, while death decreases it by one. Writing $q=1-p$, the transition probabilities on ${\mathbb{Z}}_{\ge 0}$ areand all other entries are zero. In particular $P(0,1)=1$ and $d_0=0$. The row sum is $[kp+\theta +kq]/(k+\theta )=1$. Every neighboring pair communicates, so this is an irreducible Markov chain, specifically a birth-death-immigration jump chain.

The state-zero stationary distribution equation is ${\pi }_0=q{\pi }_0+q{\pi }_1$, so ${\pi }_1=(p/q){\pi }_0$. The state-one equation is ${\pi }_1=p{\pi }_0+q{\pi }_2$, giving ${\pi }_2=(p/q){\pi }_1$. More generally, the stationary probability flow across the cut $\{0,\dots ,i\}$ is balanced, so $p{\pi }_i=q{\pi }_{i+1}$. This also follows inductively from ${\pi }_i=p{\pi }_{i-1}+q{\pi }_{i+1}$.

Hence ${\pi }_i=(p/q{)}^i{\pi }_0$. Since $p<q$, the geometric series is summable, and normalization givesThese probabilities satisfy detailed balance and therefore all the stationarity equations. The Markov chain is an irreducible Markov chain, so this normalized stationary distribution is unique.

A finite irreducible aperiodic Markov chain converges to its unique stationary distribution from every initial state. Finiteness holds because nonnegative entries with fixed finite row sums are bounded. The given connecting-sequence property makes the chain irreducible: each feasible step in such a sequence has positive proposal probability and is accepted. The all-zero index table occurs with probability $1/L>0$ and gives a self-loop at every state, establishing aperiodicity. Reversibility supplied the stationary uniform law in (b). ConsequentlyThe conclusion also holds when the feasible state space is a singleton.

Choose $k$ distinct coordinates uniformly for the starting ones; each element of $S_k$ then has starting probability $1/\left(\genfrac{}{}{0ex}{}{n}{k}\right)$. At each iteration choose one current one and one current zero uniformly and swap them. This implements the normalized proposal distribution from (c)(i), remains in $S_k$, and needs just one precomputed matrix inverse.

mymcmc <- function(x, k, sigma) {
  n <- length(x)
  stopifnot(length(k) == 1L, is.finite(k), k == floor(k),
            k > 0L, k < n, is.matrix(sigma),
            all(dim(sigma) == c(n, n)))
  sigma_inv <- invsigma(sigma)
  u <- integer(n)
  u[sample.int(n, k, replace = FALSE)] <- 1L
  draws <- matrix(0, 100L, n)
  for (iteration in seq_len(100L)) {
    ones <- which(u == 1L)
    zeros <- which(u == 0L)
    i <- ones[sample.int(length(ones), 1L)]
    j <- zeros[sample.int(length(zeros), 1L)]
    v <- u
    v[i] <- 0L
    v[j] <- 1L
    alpha <- mylogapp(x, u, v, sigma_inv)
    if (runif(1L) < alpha) u <- v
    draws[iteration, ] <- u
  }
  colMeans(draws)
}

Sampling positions with sample.int and then indexing into ones or zeros also works when $k=1$ or $n-k=1$. In R, a call such as sample(ones, 1) would incorrectly sample from 1:ones when ones is a single numeric value greater than one.

The code records the state after each of exactly $100$ transitions, including repetitions after rejection. Its answer is the vector Monte Carlo estimatorThis estimates the posterior mean coordinate by coordinate; because each coordinate is binary, its entries estimate the posterior probabilities of inclusion. Every stored state has coordinate sum $k$, so the estimated means lie in $[0,1]$ and sum to $k$.

For neighbouring states, the symmetry of the proposal and the Metropolis–Hastings acceptance probability give detailed balance:Thus the posterior distribution is a stationary distribution. Any two states in $S_k$ can be connected by moving the ones that occupy the wrong positions, and the multivariate normal distribution gives strictly positive target masses. The resulting Markov chain is therefore an irreducible Markov chain, and its long-run state averages estimate the posterior mean.

The requested $100$ iterations are a finite approximation, without a convergence guarantee or discarded burn-in. A randomly chosen starting subset has the prior distribution, rather than generally the posterior distribution. No aperiodicity assumption is needed for the finite-state Markov chain ergodic theorem for averages; indeed a uniform two-state target with $n=2,k=1$ gives a period-two alternating chain. This does not invalidate either the code or the state-average estimator.

A product-Poisson stationary distribution is suggested by balancing the three reaction fluxes around the directed cycle of reaction complexes. If their mean counts are $a,b$, this complex balance condition iswhich gives $a=4$ and $b=1/2$. This suggests the joint probability mass functionIt is normalized because it is the product of two Poisson distributions. We verify it directly in the chemical master equation, rather than assuming a product-Poisson stationary distribution theorem. The three incoming terms satisfyWith negative-index values set to zero, these identities also hold on the axes. Their sum equals $(2+n/2+nm)\pi (n,m)$, precisely the outgoing probability flux. The stationary average total exit rate is $2+{\mathbb{E}}_{\pi }n/2+{\mathbb{E}}_{\pi }(nm)=6<\infty$, so these stationary balance equations do define an invariant distribution of a continuous-time Markov chain for the nonexplosive model verified below. This is complex balance; it does not assert pairwise detailed balance for the irreversible chemical reactions.

To identify this invariant distribution with the limiting probability distribution in the question, first check that the model is a nonexplosive continuous-time Markov chain. On any finite time interval, $A_1$ is bounded by its initial count plus the finite number of arrivals in a rate-$2$ Poisson process. Conditional on that bound, the rate of producing $A_2$ is bounded, so only finitely many such productions occur. The number of joint removals cannot exceed the available molecules. Thus only finitely many chemical reactions occur in finite time.

The continuous-time Markov chain is also an irreducible Markov chain. From a Markov chain state with both counts positive, repeated joint removals reduce both. On the $A_2=0$ axis, producing one $A_2$ and then removing a pair reduces $A_1$ by one. On the $A_1=0$ axis, adding one $A_1$ and then removing a pair reduces $A_2$ by one. Each prescribed finite sequence has positive probability, so every Markov chain state reaches $(0,0)$. Conversely, arrivals followed by productions reach any $(n,m)$ with $n>0$; to reach $(0,m)$, reach $(1,m+1)$ and remove one pair. The continuous-time Markov chain is therefore positive recurrent, and the continuous-time Markov chain convergence theorem gives $\varphi =\pi$ from the empty initial condition.

The two Poisson factors happen to have cancelling numerical coefficients at $(2,2)$:

For positive rates the three states communicate, so the finite continuous-time Markov chain has a unique stationary probability vector. The first and third components of $W{p}^{\mathrm{st}}=0$ giveThese two edge-balance relations and normalization yield, with $D=f_3{f}_4+f_1{f}_3+f_1{f}_2$,Their sum is one and each is nonnegative. The middle balance equation then follows from the other two, as expected from probability conservation. A line graph has no cycle that could carry a nonzero stationary circulation: each edge's forward and backward stationary traffic balances even if the bare rates differ. If some rates vanish, the process need not be an irreducible Markov chain and its stationary distribution need not be unique. The displayed expression remains valid when its denominator is nonzero and its limits select the communicating class reached, but it must not be used as a unique late-time law when disconnected closed classes retain dependence on the initial state.

For finite irreducible Markov chains that are aperiodic Markov chains and reversible Markov chains, the ratio definition of cutoff and $T_n=t_{\mathrm{mix}}^{(n)}(1/4)\to \infty$ imply the Peres product condition, with ${\gamma }_n=1-{\lambda }_{2,n}$. If ${\gamma }_n{T}_n\le B$ on a subsequence, then ${\gamma }_{\ast ,n}\le {\gamma }_n$ and the eigenvalue lower bound for total variation mixing giveFor sufficiently small fixed $\epsilon$, this contradicts the limit one forced by cutoff for Markov chains. The same argument proves the stronger conclusion ${\gamma }_{\ast ,n}{T}_n\to \infty$.