Past exam of the options I course of the University of Oxford / 2012 / M (Probability)

The sequence is a Markov chain when the conditional law of its next state depends on the past only through the current state. Precisely, for each history of positive probability and every $j\in S$,A time-homogeneous chain additionally has transition probabilities independent of $n$, written $P_{ij}=\mathrm{Pr}(X_{n+1}=j\mid X_n=i)$. The displayed single transition matrix in the subsequent parts specifies this time-homogeneous case.

First the entries must be nonnegative: together with the given $a,b\ge 0$, validity of the transition matrix requires $0\le a\le 1$ and $0\le b\le 1/3$. Every row then sums to one.

The only incoming edge to state $2$ is $1\to 2$, of probability $a$. If $a=0$, no other state can reach $2$, so the chain is not irreducible. If $a>0$, all edges in the directed cyclehave positive probability, since $P_{23}=1/3+b>0$, $P_{34}=1$ and $P_{41}=1/2$. Traversing this cycle connects every state to every other state. Therefore

Within the valid parameter rectangle, suppose $a>0$ and $b<1/3$. The chain is then irreducible by part (i). State $1$ has a two-step return $1\to 2\to 1$ and a three-step return $1\to 2\to 4\to 1$, both of positive probability. Its period of a state in a Markov chain therefore divides both two and three, hence equals one. Irreducibility gives the same period to every state, so the chain is aperiodic.

If $b=1/3$ and $a>0$, every transition goes between the two classes $\{1,3\}$ and $\{2,4\}$. Returns must have even length, while $1\to 2\to 1$ has length two. Thus the irreducible chain has period two.

If $a=0$, the closed communicating class $\{1,3,4\}$ alternates between $\{1,3\}$ and $\{4\}$. Its states have period two, using the positive two-step returns through $4$. Consequently the chain is not aperiodic, regardless of how the transient state $2$ is classified. Combining the cases,

Here the valid parameters lie in the irreducible aperiodic range. The finite-state theorem on convergence to an invariant distribution therefore implies convergence to the unique stationary distribution, independently of the initial state.

Solve $\pi P=\pi$ and ${\sum }_i{\pi }_i=1$. The equations for states $1,2,3$ giveHence ${\pi }_3={\pi }_1$ and ${\pi }_4=5{\pi }_1/3$. Normalization gives ${\pi }_1(1+1/2+1+5/3)=1$, soStationarity alone would not guarantee convergence of the one-time distributions; irreducibility and aperiodicity justify that final limit.

A joint probability density function must be measurable, nonnegative almost everywhere, and normalized:It determines the joint law by $\mathrm{Pr}((X,Y)\in B)={\int }_{B}f(x,y)dxdy$ for measurable sets $B$. Values on sets of planar measure zero may be changed without changing that law.

The variables are not independent. Their joint support excludes $X>1/2$ and $Y>1/2$ simultaneously, since every point with positive joint density satisfies $X+Y\le 1$. Yet each event separately has positive probability: the density is strictly positive on open triangular subregions where $X>1/2$, and similarly where $Y>1/2$. ThusThis contradicts the event factorization required for independence of random variables. The product-shaped formula $24xy$ on its support does not undo the dependence created by the triangular support.

For $0\le x\le 1$, integrate the joint probability density function over its allowed $y$-range:Outside this interval it is zero. Therefore the marginal density isIts integral is $12(1/2-2/3+1/4)=1$, also checking the original joint normalization. By symmetry $Y$ has the same marginal density.

From the marginal density,The mixed moment isThus the covariance isThe negative sign is consistent with the support constraint: a large value of one variable leaves a smaller available range for the other.

For $0<x<1$, the conditional density is obtained by dividing the joint density by the positive marginal density:Consequentlywhich integrates to one. Its conditional expectation isThis uses the usual continuous conditional-density version, defined by the density ratio at a point where $f_X$ is positive; conditioning does not require the event $X=1/2$ itself to have positive probability.