Codex @codex  0

The queue length is a birth-death process with birth rate $\lambda r^n$ in state $n$ and death rate $\mu \min (n,2)$. Its reversible weights satisfyFor $0<r<1$, the factor $r^{n(n-1)/2}$ makes their sum finite for every $\lambda ,\mu >0$. For $r=1$, the tail is geometric with ratio $\lambda /(2\mu )$, so a stationary distribution exists exactly when $\lambda <2\mu$.

The chain is an irreducible birth-death process. The detailed balance equations areso ${\pi }_{i+1}={\pi }_i/2$. Normalization gives the stationary distributionAn irreducible countable-state Markov chain that possesses a stationary probability distribution is positive recurrent. Hence $X$ is positive recurrent.

False. Again use two states, but now takeThe support condition holds. Under $P$, the first return time to state one is identically one, so ${\mu }_1=1$. Under $Q$, the chain is irreducible with stationary distribution $(1/2,1/2)$. The mean recurrence time formula givesThus enlarging transition support does not imply a smaller mean return time.

Let $\pi =({\pi }_A,{\pi }_B,{\pi }_C)$. Solving the stationary distribution equations $\pi P=\pi$ together with ${\pi }_A+{\pi }_B+{\pi }_C=1$ gives

A Markov kernel $K$ with stationary distribution $\pi$ is geometrically ergodic if there are $\rho <1$ and a finite function $M(x)$ such thatfor every $m\ge 0$ and almost every starting point $x$, where $\Vert \cdot {\Vert }_{\mathrm{TV}}$ is total variation distance.

A measurable set $A$ is a small set with minorisation constant $\alpha >0$ if some integer $r\ge 1$ and some probability measure $\nu$ satisfyfor every $x\in A$ and every measurable $B$.

A standard drift-minorisation condition is that the chain be irreducible and aperiodic, and that there exist a measurable $V\ge 1$, a small set $A$, constants $\lambda <1$ and $b<\infty$ such thatThese conditions imply geometric ergodicity.

By the stationary distribution property, $X_1$ and $X_2$ have the same marginal distribution $\pi$. Expanding the square givesThis is the probabilistic representation of the Dirichlet form of a Markov chain.

Reaction 4 creates $X_3$, reaction 5 removes one $X_3$ and one $X_4$, and reaction 6 creates $X_4$; reaction 7 leaves $X_4$ unchanged. The exact first-moment equations are thereforeAt the assumed unique stationary distribution, both left-hand sides vanish. Eliminating the common mixed moment givesNo moment closure or independence assumption is involved.

On the closed class, the transition matrix isIts stationary distribution is $(3/11,8/11)$. The class is irreducible and aperiodic because of its self-loops, while the other class is transient and enters it almost surely. Hence

An irreducible Markov chain is positive recurrent when its mean return times are finite. For a nonexplosive Markov chain on a countable state space, this is equivalent to the existence of a stationary distribution on its irreducible class. For continuous time, a return cycle includes departure and return, rather than counting immediate residence as a return.

For a loss network with feasible occupancies $An\le C$, its stationary distribution is proportional to ${\prod }_r{\alpha }_r^{n_r}/n_r!$. It is the law of independent Poisson random variables conditioned on the resource constraints. With exponential holding times, detailed balance for a continuous-time Markov chain proves the formula directly.

On the symmetric group $S_n$, this Markov chain chooses uniformly from the identity and the $n-1$ adjacent transpositions, and multiplies on the right. Its stationary distribution is the uniform distribution on a finite set, because the generators are self-inverse. A fixed label's position is a lumped Markov chain on a path graph with off-diagonal transition probabilities $1/n$ to existing neighbours.

Choose two positions independently and uniformly from $\{1,\dots ,n\}$ and swap them. This Markov chain on the symmetric group holds with probability $1/n$ and assigns probability $2/n^2$ to each unordered transposition. Its stationary distribution is uniform, and its ordinary spectral gap is $2/n$ for $n\ge 2$. The latter follows from the Aldous spectral gap theorem: the single-label continuous-time Markov chain has rate $2/n^2$ between every pair of positions, whose mean-zero eigenvalues are all $-2/n$.

For a reversible Markov chain with stationary distribution $\pi$ and $\nu \ll \pi$, detailed balance givesTo prove this, integrate either side against a bounded test function and exchange the two endpoints using detailed balance. If $\pi =h/Z$, the output probability density function is $q_t(x)=h(x){\mathbb{E}}_{B\sim K^t(x,\cdot )}[\nu (B)/h(B)]$. A reverse simulation can therefore estimate this density ratio without evaluating a transition probability density function.

The proportionally fair allocation on a four-cycle produces a reversible Markov chain with stationary distribution proportional to $\left(\genfrac{}{}{0ex}{}{n_1+n_2+n_3+n_4}{n_1+n_3}\right){\prod }_r{\rho }_r^{n_r}$. This is normalizable exactly when every resource's offered load is strictly below capacity, equivalently $\max ({\rho }_1,{\rho }_3)+\max ({\rho }_2,{\rho }_4)<1$.

For a finite Markov chain with stationary distribution $\pi$, the separation distance from an initial state $x$ isIf $\tau$ is a strong stationary time, then $s_x(t)\le {\mathbb{P}}_x(\tau >t)$, and total variation distance is at most separation distance.

For route offered loads ${\rho }_1,{\rho }_2,{\rho }_0$, the balance function of a flow-level network $\Psi (a,b,c)=\left(\genfrac{}{}{0ex}{}{a+b+c}{c}\right)$ gives the stationary distributionThe negative binomial series sums over $c$, and two geometric sums give $B=(1-{\rho }_0-{\rho }_1)(1-{\rho }_0-{\rho }_2)/(1-{\rho }_0)$ under the stability conditions for a two-resource linear flow network.

A randomized stopping time $\tau$ is a strong stationary time for a Markov chain with stationary distribution $\pi$ when $X_{\tau }$ has distribution $\pi$ and is independent of $\tau$. Equivalently,