The Poisson process gives $N_k\sim \mathrm{Pois}(k)$, with expected value and variance both equal to $k$. Consequently $Y_k=(N_k-k)/\sqrt{k}$. Scaling and translating the argument in the moment-generating function of a Poisson distribution gives, for every real $t$,The centering term must remain inside the exponent: omitting it would leave a divergent first-order contribution.

Complete the square in the integral against the standard normal density:The remaining integral is one because its integrand is the probability density function of the normal distribution $N(t,1)$. Hence the moment-generating function of a standard normal variable is finite on the whole real line.

Using the probability mass function of the Poisson distribution and the power series of the exponential function,This moment-generating function of a Poisson distribution is finite for every real $t$. It also covers $\mu =0$, when $X=0$ with probability one and the moment-generating function is identically one.

The moment-generating function of a real random variable $X$ isfor real $t$ for which this expected value is finite. Because the integrand is nonnegative, the extended expected value is always defined, but it may equal $+\infty$. We always have $M_X(0)=1$; the usual uniqueness and convergence theorems require finiteness on an open interval containing zero. When this condition holds, derivatives at zero recover the moments, $M_X^{(r)}(0)=\mathbb{E}[X^r]$.

Put $Z_n=n(1-M_n)$. Its survival function isFor each fixed $z\ge 0$, eventually $n\ge z$, and the exponential limit gives $(1-z/n{)}^n\to e^{-z}$. Thus its cumulative distribution function tends to $0$ for $z<0$ and to $1-e^{-z}$ for $z\ge 0$. This continuous limiting cumulative distribution function identifies the exponential distribution with rate one:This is the exponential endpoint limit of a uniform maximum: the gap from the upper endpoint is typically of order $1/n$, although the maximum itself approaches a constant.

The sample maximum has cumulative distribution functionThe product $x^n$ follows from independence of random variables and the continuous uniform distribution of each observation. For every $x<1$, this cumulative distribution function tends to zero; for $x>1$ it equals one. These are exactly the values of the cumulative distribution function of the constant random variable $1$ at its continuity points. Therefore, by the definition of convergence in distribution,At $x=1$ the limiting cumulative distribution function is discontinuous, so the definition does not require checking that point, although $F_{M_n}(1)=1$ also holds.

This is the synchronous product of simple symmetric random walks, rather than a walk that changes only one coordinate at a time. Independence of the three coordinate processes givesOdd-time return probabilities vanish. The p-series with exponent $3/2$ converges, so the sum of all return probabilities is finite. By the recurrence criterion by return probabilities, the origin is a transient state: the three-dimensional process is transient. From the origin its communicating class consists of integer triples whose coordinates have the same parity; translation gives the same classification for every other class.

For the symmetric simple random walk on the integer line, odd return times are impossible. A return after $2m$ steps requires exactly $m$ positive increments, soby Stirling's approximation. The resulting sum diverges by the p-series criterion. The recurrence criterion by return probabilities therefore makes zero a recurrent state. The simple random walk is an irreducible Markov chain on $\mathbb{Z}$, so the symmetric chain is recurrent.

Represent the simple random walk by $X_n={\xi }_1+\cdots +{\xi }_n$, where the increments are independent and identically distributed and satisfy $\mathbb{P}({\xi }_k=1)=p$, $\mathbb{P}({\xi }_k=-1)=1-p$. They are integrable and have expectation $2p-1$, so the strong law of large numbers givesIf $p\ne 1/2$, this nonzero drift implies that $X_n$ eventually leaves every fixed finite interval and tends to the appropriate infinity. Hence it visits zero only finitely many times almost surely, so zero is a transient state. By translation of the starting state, every state is transient: the chain is transient for $p\ne 1/2$. This also covers the deterministic endpoint cases $p=0,1$, without assuming Markov chain irreducibility at those endpoints.

The first Markov chain is a finite irreducible Markov chain: its positive-probability directed cycle reaches all four states. The self-loop at state 1 makes it an aperiodic Markov chain. Convergence to an invariant distribution therefore givesThe second chain alternates between the sets $\{1,3\}$ and $\{2,4\}$. It is a periodic Markov chain with period two, so $p_{11}^{(2m+1)}=0$. From either state 2 or state 4 the next-step probability of entering state 1 is $2/3$. Therefore $p_{11}^{(2m)}=2/3$ for every $m\ge 1$. The subsequences have different limits, so the second return-probability sequence does not converge.

Write the stationary distribution as a row vector $\pi =(a,b,c,d)$. For the first transition matrix, $\pi P=\pi$ gives $a=a/2+d$, $b=a/2+b/2$, $c=b/2$ and $d=c$. Hence $b=a$ and $c=d=a/2$. Normalization yieldsBy the stated common-stationary distribution property, this is also the answer for the second transition matrix.

The inverse change of variables is $y_1=uv$, $y_2=(1-u)v$, on $0<u<1$, $v>0$. Its absolute Jacobian determinant isThe change of variables for probability densities therefore givesThis factorizes on a product domain into a unit uniform density in $u$ and a shape-two Gamma density in $v$. Thus $U$ and $V$ are independent, with $U\sim \mathrm{Uniform}(0,1)$ and $V\sim \mathrm{Gamma}(2,\lambda )$. This is the shape-$(1,1)$ case of beta-gamma independence.

The Poisson interarrival times of a homogeneous Poisson process are independent and identically distributed with the exponential distribution of rate $\lambda$. Consequently $Y_1,Y_2$ are independent random variables and their joint probability density isHere $Y_2$ is the second interarrival duration, rather than the time of the second arrival.

Now $M_{n+1}=\max (M_n,X_{n+1})$. Again $X_{n+1}$ is independent of the whole previous history, so the recursion is a Markov chain obtained from independent innovations. To remain at $s$, the new observation must be at most $s$; to jump to $t>s$, it must equal $t$. Thus the transition probabilities areThe upper tail sums to $1/(s+1)$, complementing the holding probability $s/(s+1)$.

Write $S_{n+1}=S_n+X_{n+1}$. The new random variable $X_{n+1}$ is independent of the entire history generated by $X_1,\dots ,X_n$, hence of $(S_1,\dots ,S_n)$. This is a Markov chain obtained from independent innovations: conditional on the past, only the current sum enters the next state. Its time-homogeneous transition probabilities areEvery jump is a positive integer, and each row sums to one by the probability mass function from part (b).

For $x>0$, the event $M_n/n\le x$ says that each of the independent random variables is at most $\lfloor nx\rfloor$. Hence, for all sufficiently large $n$,Since $n/(\lfloor nx\rfloor +1)\to 1/x$ and $\log (1-u)=-u+o(u)$ as $u\to 0$, taking logarithms gives the limit $e^{-1/x}$. For $x\le 0$ the probability is zero. ThereforeThis is the Fréchet limit for a reciprocal integer tail, with a Fréchet distribution of shape one. Its cumulative distribution function is continuous even at zero, so the pointwise calculation establishes convergence in distribution.

Subtract adjacent values of the cumulative distribution function. Its value at zero is zero, so the same calculation applies also to $k=1$:The probability mass function sums to one by a telescoping series.

The implication convergence in probability implies convergence in distribution follows by comparing nearby events. Fix a continuity point $x$ of the cumulative distribution function $F_X$ and let $\epsilon >0$. If $X\le x-\epsilon$ and $|X_n-X|\le \epsilon$, then $X_n\le x$. Conversely, $X_n\le x$ and $|X_n-X|\le \epsilon$ imply $X\le x+\epsilon$. ThusThe error probability tends to zero by convergence in probability. Taking lower and upper limits and then letting $\epsilon \downarrow 0$, continuity at $x$ squeezes both bounds to $F_X(x)$. This is exactly convergence in distribution.

Let $\mu =\mathbb{E}[X_1]$ and ${\sigma }^2=\mathrm{Var}(X_1)\in (0,\infty )$. For independent and identically distributed random variables, the central limit theorem statesThe limit is the standard normal distribution. If the variance is zero, then $X_i=\mu$ almost surely; the centered sums are zero, and division by $\sigma$ is not defined.

Convergence in distribution means that the cumulative distribution functions satisfyat every $x$ where the cumulative distribution function of $X$ is continuous. Values at its discontinuities need not converge. This definition uses only the probability distributions, so the random variables need not be defined on the same probability space.