Past exam of the options I course of the University of Oxford / 2009 / M (Probability)

Pointwise,Taking expectations and dividing by $a^2>0$ gives the second-moment form of Markov's inequality:

The inverse transformation is $x=uv$, $y=u(1-v)$, with support $u>0$, $0<v<1$. Its absolute Jacobian determinant isThe original joint probability density function, by independence, is ${\lambda }^2{e}^{-\lambda (x+y)}$ on the positive quadrant. The change of variables formula therefore givesIntegrating over $v$ givesthe gamma distribution of shape two and rate $\lambda$. Integrating over $u$ gives $f_V(v)=1$ on $(0,1)$. The factorization also shows that $U$ and $V$ are independent.

The preceding marginal calculation makes each $V_i$ uniformly distributed on $(0,1)$, with expectation $1/2$ and variance $1/12$. By independence,Apply part (a) to the centred random variable $Z=S_n/n-1/2$ to obtainThis is Chebyshev's inequality in this setting.

For $n=0$, the conditional transition probabilities are ${\delta }_{ij}$, the entries of $P^0=I$; for $n=1$ the assertion is the definition of $P$. For the induction step, condition on the state at time $n$ and use the Markov property and time-homogeneity:The countable sum is justified by the law of total probability; all terms are nonnegative. Thus the formula follows for every $n$ by induction. It is the Chapman-Kolmogorov equation for repeated one-step transitions.

Let $u_n=\mathbb{P}(X_n=1\mid X_0=1)$. Conditioning on $X_n$ givesThe fixed value is $b/(a+b)$. Subtracting it and iterating the scalar recurrence yieldsThe formula includes $n=0$. If $a+b=1$, its value at $n=0$ is interpreted directly as $u_0=1$, while $u_n=b$ for all $n\ge 1$. Since $|1-a-b|<1$, the limiting probability is $b/(a+b)$.

At the first step, the chain enters state two with probability $p$ and state four with probability $1-p$. The classes $\{2,3\}$ and $\{4,5\}$ are closed and disjoint, so only the first branch can reach state three. On $\{2,3\}$ the transition matrix is exactly the two-state matrix from part (b). Thus after the remaining $n-1$ steps, the probability of the second state in that class isFor $n=1$ it is zero, as the original matrix requires; this statement covers the zero-base exponent case without ambiguity.

State one is left after one step and never revisited. The eventual probability mass in the closed class $\{4,5\}$ isWithin this class the stationary distribution is $(b/(a+b),a/(a+b))$, because the balance equation is $a{\pi }_4=b{\pi }_5$. The two-state recurrence shows convergence from either starting state, since $|1-a-b|<1$. The other closed class never reaches state five. Therefore