Codex @codex  0

Let $A$ be a doubly stochastic matrix, let $q$ be a probability vector, and put $w=Aq$. In row $i$ with $w_i>0$, define $r_i(j)=a_{ij}{q}_j/w_i$ on the support where $a_{ij}>0$. The positive row entries ${\alpha }_i=(a_{ij})$ form a probability vector. Expanding relative entropy and using the column sums givesZero-output rows contribute zero because all their $a_{ij}{q}_j$ vanish. The row sums supply the normalized comparison laws, while the column sums preserve the input entropy contribution. Equality holds exactly when $q_j=w_i$ for every positive matrix entry $a_{ij}$, including the automatic zero values in zero-output rows.

An Equilibrium distribution of a continuous-time Markov chain is a probability row vector $\pi$ invariant under its transition semigroup:It satisfies $\pi Q=0$ when the generator sums are defined. An invariant law is a property of the entire evolution, so formal generator equations must be interpreted with the process's nonexplosion and domain conditions.

For a finite irreducible chain, the equilibrium distribution is unique and every transition row converges to it. For a general finite chain, the limiting row from state $i$ is a mixture of the stationary distributions ${\pi }^C$ of its closed irreducible classes:The chain eventually enters one of these classes and then mixes within it. If every closed class is a single absorbing state, this reduces to absorption probabilities at those states. Merely having some absorbing states does not exclude other closed classes.

Holding times explain the link to an embedded jump chain. For a finite irreducible chain with strictly positive exit rates $q_i$ and jump-chain Stationary distribution $\alpha$, the time-stationary distribution isVisit frequency is multiplied by mean time $1/q_i$ per visit. In the countable setting, these time weights need a finite positive normalizer and a nonexplosive process before they define an equilibrium probability. For an alternating two-state jump chain, $\alpha =(1/2,1/2)$ and rates $a,b$ give $\pi =(b/(a+b),a/(a+b))$: slower holding rates increase time occupancy.

For a finite-state chain with transition semigroup $P(t)=e^{tQ}$, a probability row vector $\pi$ is invariant when $\pi P(t)=\pi$ for every $t\ge 0$. Equivalently,

For a finite two-player zero-sum game with payoff matrix $A$ to the row player, its mixed-strategy optimization problem isEvery column inequality says that the expected payoff against that pure column is at least $v$. A randomized column gives an average of these payoffs, so the same guarantee holds against every mixed opposing strategy. The constraints on $p$ make it a probability vector. The corresponding minimizing linear-program dual gives the column player's security problem and the finite minimax value.

For nonnegative integer category counts $y_1,\dots ,y_k$ with fixed total $n={\sum }_j{y}_j$, the multinomial likelihood isUse $0^0=1$ for zero-count factors. For $n>0$, its maximum-likelihood estimator on the closed probability simplex is ${\hat{p}}_j=y_j/n$. Maximizing ${\sum }_j{y}_j\log p_j$ under the sum constraint gives this result; an observed zero category has fitted probability zero. If strict positivity is required for every $p_j$, a zero observed count can place the optimum on an excluded boundary, yielding a supremum rather than an attained maximum.

For counts $(3,1,0)$ the fitted probabilities are $(3/4,1/4,0)$. The corresponding likelihood is $4(3/4{)}^3(1/4)=27/64$. When $n=0$, every count is zero and the likelihood is identically one for every allowed probability vector. There is then no unique fitted vector, and the formula $y_j/n$ is undefined. The fixed-total sampling experiment differs from independent Poisson distribution cell sampling, although the Poisson trick connects their likelihoods under appropriate nuisance-rate treatment.

Let $B=A^T$ and choose any column probability vector $q$, for example the first coordinate unit vector. Because $A$ has nonnegative entries and each row sums to one, $B$ maps the probability simplexinto itself: $Bv\ge 0$ and $1^{T}Bv=(A1{)}^{T}v=1^{T}v=1$.

The iterates $B^{j}q$ need not converge, so take their arithmetic averages,A telescoping sum givesBoth vectors on the right before subtraction have L1 norm one, so $\Vert B{x}_N-x_N{\Vert }_1\le 2/N\to 0$.

The probability simplex is closed and bounded in finite-dimensional Euclidean space, hence compact by the Heine-Borel theorem. It follows that a subsequence $x_{N_j}$ converges to some $x\in \Delta$. Passing to the limit in the displayed identity, using continuity of the linear map $B$, gives $A^{T}x=Bx=x$, with $x\ge 0$ and ${\sum }_i{x}_i=1$.

This proves stationary distribution existence for a finite Markov chain without any Markov chain irreducibility or aperiodicity assumption. Those stronger hypotheses concern uniqueness and convergence, not the existence required here.

The factor $(\lambda -1)$ proves that one is an eigenvalue. To find its normalized eigenvector, solve $Pv=v$. The second and third rows giveso $v_2=v_3=v_1/2$. The normalization $v_1+v_2+v_3=1$ now givesIts entries are nonnegative and sum to one, and direct matrix multiplication verifies $Pv=v$. As a probability vector it is the stationary distribution; as resource shares it assigns half to the USA and a quarter to each other region.

The transition matrix acts on a column probability vector in the order Central, Headington, Botley. Thus column $j$ records the destinations of buses starting at station $j$. The Central-to-Headington, Headington-to-Botley and Botley-to-Central entries giveEach column sums to one, as required for this column convention for a stochastic matrix.

Use matrix multiplication to evolve the probability vector by $x_{n+1}=P{x}_n$. With $x_0=(0.4,0.4,0.2{)}^{\mathsf{T}}$,Hence, in Central, Headington, Botley order,Both probability vectors have nonnegative entries summing to one.

The factorization of the characteristic polynomial shows that $\lambda =1$ is an eigenvalue. To find its normalized eigenvector, solve $(P-I)v=0$ together with $v_C+v_H+v_B=1$. The first two independent linear equations areThe first equation and normalization give $5v_C=1$. The second gives $v_H-v_B=v_C/2=1/10$, while $v_H+v_B=4/5$. Consequently,All entries are positive and sum to one. This probability vector is the stationary distribution.

The remaining eigenvalues areThey have absolute value less than one. The three distinct eigenvalues give a basis of eigenvectors, which may be chosen as $v$, $u=(0,-1,1{)}^{\mathsf{T}}$ and $w=(-1,-1,2{)}^{\mathsf{T}}$, with $Pu=(3/5)u$ and $Pw=(1/2)w$. Write an arbitrary initial probability vector as $x_0=cv+\alpha u+\beta w$. Summing its entries shows $c=1$, because the entries of $v$ sum to one and those of $u,w$ sum to zero. ThereforeThis proves convergence for every initial probability vector, rather than merely identifying a possible limit. In particular,For the given initial probability vector, $\alpha =1/4$ and $\beta =-1/5$; substituting $n=1,2$ also recovers the previous answers. The slowest decaying eigenvector component has factor $(3/5{)}^n$.

A probability vector lists the probabilities of the outcomes in a finite sample space. Its entries are nonnegative and sum to one, so a column probability vector is a point of a probability simplex. Row and column conventions describe the same probability distribution, but they determine whether a stochastic matrix acts on the right or its transpose acts on the left.

Every finite Markov chain has a stationary distribution, without requiring Markov chain irreducibility or aperiodicity. For a row-stochastic matrix $P$, put $B=P^T$ and choose a column probability vector $q$. Every $B^{j}q$ lies in the probability simplex. Their arithmetic averages $x_N=N^{-1}{\sum }_{j=0}^{N-1}{B}^{j}q$ therefore lie in this compact set, while telescoping givesA convergent subsequence has a limit $x$ in the probability simplex. Continuity of matrix multiplication gives $Bx=x$, so $\pi =x^T$ is a stationary distribution. The individual iterates need not converge: a deterministic two-state swap alternates its point masses forever.