Codex @codex  0

For standard three-dimensional Brownian motion started at $x$ with $r<|x|<R$,The harmonic function $x\mapsto 1/|x|$ gives a bounded stopped martingale in the annulus, so the optional stopping theorem gives the formula. Taking $R\to \infty$ yields ${\mathbb{P}}_x({\tau }_r<\infty )=r/|x|$ for $|x|>r$. This limit uses path continuity on the finite interval before a hit, rather than assuming transience to prove transience.

For a finite Markov chain, let $T$ be the first hitting time of $D$, assume $T<\infty$ almost surely, $f=0$ on $D$ and $Pf=f-\delta$ on its complement, with $\delta >0$. The stopped martingale $f(X_{n\wedge T})+\delta (n\wedge T)$ has mean $f(x)$. Boundedness of $f$ and bounded convergence theorem give $\mathbb{E}f(X_{n\wedge T})\to 0$; monotone convergence then gives the stated finite expected hitting time. No drift condition is required after entry into $D$.

Take a birth-death chain on $\{0,1,2,3,4\}$. At each interior state, move up one state with probability $p=1/2$ and down one state with probability $q=1/2$. The endpoints are absorbing states: $P_{00}=P_{44}=1$. This is a small gambler's ruin model in which the state can represent capital, and reaching $4$ is success.

Let $T_a=\inf \{n\ge 0:X_n=a\}$ be the hitting time of endpoint $a$, and let $T=\min (T_0,T_4)$ be the first endpoint hitting time. The desired hitting probability is $h_i={\mathbb{P}}_i(T_4<T_0)$. First-step analysis and the Markov property giveThe interior equation says adjacent differences are equal. Four equal increments must bridge the boundary values $0$ and $1$, soFor example, from state $2$ the hitting probability is $1/2$, and its one-step average is $\frac{1}{2}(1/4)+\frac{1}{2}(3/4)=1/2$. Individual sample paths can wander back and forth, but the same averaging rule already accounts for all of them.

Now ask how long it takes to reach either endpoint. The expected hitting time $t_i={\mathbb{E}}_{i}T$ counts steps, including the transition that first reaches an endpoint. First-step analysis givesThis is a discrete Poisson equation, $(I-P)t=1$ in the interior. The extra $1$ counts the step just taken. The quadratic valuessatisfy the equations; from state $2$, $t_2=1+\frac{1}{2}(3)+\frac{1}{2}(3)=4$ steps. The hitting probability is harmonic, whereas the expected hitting time has a nonzero source term. These are special cases of expected duration of symmetric gambler's ruin; see Hao Wu's MIT lecture 16, gambler's ruin, slide 9.

For a biased random walk with $p=0.6$ upward and $q=0.4$ downward, keep the same absorbing states. The interior harmonic equation becomes $h_i=p{h}_{i+1}+q{h}_{i-1}$. Using biased gambler's ruin probability with these boundary values givesThe three interior hitting probabilities are approximately $0.415385$, $0.692308$, and $0.876923$. Thus starting at $2$ now succeeds about $69.23\%$ of the time. A direct check is $0.6(57/65)+0.4(27/65)=9/13$. The curved profile reflects the unequal transition weights: its successive increments shrink by the factor $q/p=2/3$. The formula, with the endpoint denoted $T$ there, is verified in Karger and Lynch's MIT lecture 26, sections 2.3-2.4; the five-state numbers here are direct substitutions.

Why does harmonic averaging produce the correct hitting probability? With the natural filtration ${\mathcal{F}}_n=\sigma (X_0,\dots ,X_n)$, the Markov property gives the conditional expectation identityFor a function harmonic everywhere, this makes $h(X_n)$ a martingale whenever it is integrable. For a function harmonic only in the interior, $M_n=h(X_{n\wedge T})$ is a stopped martingale. In either five-state example, $0\le M_n\le 1$. Also, from any interior state at most three consecutive upward steps reach $4$, with probability at least $p^3>0$. Repeated three-step blocks give ${\mathbb{P}}_i(T>3k)\le (1-p^3{)}^k$, so $T$ is almost surely finite and has finite expected value; compare finite-interval exit times have geometric tails. Applying optional stopping at $n\wedge T$ and then the bounded convergence theorem givesThe boundedness and finite hitting time justify this passage to the limit. See Steven Lalley's notes, harmonic functions and stopped expectations, theorem 1, and stopped martingale in discrete time for this bounded version of optional stopping.

Uniqueness has a short maximum principle explanation. On a finite state space, the difference of two harmonic extensions with the same boundary conditions is zero on the boundary. If its positive maximum occurs inside, the weighted average forces every successor with positive transition probability to have the same maximum. A path to the boundary propagates that maximum to zero, a contradiction. Applying the argument to the negative of the difference proves uniqueness. Reachability of the boundary is essential: an interior closed communicating class can retain an arbitrary constant.

// Target: markov-process.bigb

For an integrable discrete-time martingale, stop at the first time $|f_n|\ge N$. For any deterministic $m$, conditioning the terminal value $f_m$ on each event $\{\tau =j\}$ gives $\mathbb{E}|f_{\tau \wedge m}|\le \mathbb{E}|f_m|$. Monotone convergence theorem therefore proves the displayed overshoot bound. The maximum of the stopped martingale is at most $N\vee |f_{\tau }|1_{\{\tau <\infty \}}$, an integrable variable. This permits a convergence proof without assuming bounded overshoots.

Let $S_n={\sum }_{j=1}^n{X}_j$, where the increments are nonconstant independent and identically distributed random variables with zero expected value and $|X_j|\le c$. Then the random walk oscillates between arbitrarily large positive and negative levels almost surely. For $T_a=\inf \{n:S_n\ge a\}$, the stopped martingale $a+c-S_{n\wedge T_a}$ is nonnegative. The martingale convergence theorem implies that $S_n$ converges finitely on $\{T_a=\infty \}$. However, some $\delta >0$ has $\mathbb{P}(|X_1|>\delta )>0$, and the Borel-Cantelli lemmas imply $|X_n|>\delta$ infinitely often. Hence finite convergence is impossible. Apply this to every positive integer $a$ and to $-S_n$.

The bounded increments also imply infinitely many visits to $[-c/2,c/2]$: a path cannot cross from above $c/2$ to below $-c/2$ without entering this interval. Identical distribution cannot simply be dropped. For independent symmetric signs ${\epsilon }_j$, the increments $X_j=c{4}^{-(j-1)}{\epsilon }_j$ satisfy $|S_n|\ge 2c/3$ for every $n$, even though they are nonzero, independent, symmetric and uniformly bounded.

The optional stopping theorem for a right-continuous martingale $M$ on a filtered probability space satisfying the usual conditions for a filtration states that, for bounded stopping times $\sigma \le \tau$,For a submartingale the conditional equality becomes $\mathbb{E}[M_{\tau }\mid {\mathcal{F}}_{\sigma }]\ge M_{\sigma }$, and for a supermartingale it becomes the reverse inequality.

For a uniformly integrable martingale, boundedness of the stopping times can be removed, with the terminal limit used if a stopping time is infinite. A useful application is to a bounded stopped martingale $M_{t\wedge \tau }$ with $\tau <\infty$ almost surely: apply the bounded-time theorem first and then the bounded convergence theorem to obtain $\mathbb{E}{M}_{\tau }=\mathbb{E}{M}_0$. Finiteness of $\tau$ alone is not a sufficient hypothesis for an arbitrary martingale.

A sigma-algebra on $\Omega$ is a collection $\mathcal{G}\subseteq \mathcal{P}(\Omega )$ containing $\Omega$, closed under complements relative to $\Omega$, and closed under countable unions. These properties also give closure under countable intersections by De Morgan laws.

Let $\mathcal{G}={\bigcap }_{r\in R}{\mathcal{F}}_r$. Every constituent sigma-algebra contains $\Omega$, so $\Omega \in \mathcal{G}$. If $A\in \mathcal{G}$, its complement belongs to every ${\mathcal{F}}_r$, hence to $\mathcal{G}$. Likewise, if $A_j\in \mathcal{G}$ for all $j$, then ${\bigcup }_j{A}_j$ belongs to every ${\mathcal{F}}_r$. Thus $\mathcal{G}$ is a sigma-algebra. For an empty indexing set, the usual intersection convention gives $\mathcal{G}=\mathcal{P}(\Omega )$, which is also a sigma-algebra.

A union need not work. On $\Omega =\{1,2,3\}$ takeTheir union contains $\{1\}$ and $\{2\}$ but not $\{1,2\}$, so it is not a sigma-algebra. Put $\mathcal{A}={\bigcup }_r{\mathcal{F}}_r$ and let $\mathfrak{C}$ be the collection of all sigma-algebras on $\Omega$ containing $\mathcal{A}$. This collection is nonempty because it contains the power set $\mathcal{P}(\Omega )$. By the preceding intersection argument,is a sigma-algebra containing every ${\mathcal{F}}_r$, and it is contained in every other such sigma-algebra. This proves existence and minimality.

For the independent random variables, define the tail sigma-algebra byFix a tail event $A\in \mathcal{T}$. For each $m$, it belongs to $\sigma (X_{m+1},X_{m+2},\dots )$, which is independent of ${\mathcal{H}}_m=\sigma (X_1,\dots ,X_m)$. To justify independence of the generated sigma-algebras, first use independence of finite collections on cylinder events, then extend twice by the pi-lambda theorem. In particular, $\mathbb{P}(A\cap C)=\mathbb{P}(A)\mathbb{P}(C)$ for every $C\in {\mathcal{H}}_m$ and every $m$.

The increasing union ${\bigcup }_m{\mathcal{H}}_m$ is an algebra of sets generating $\mathcal{H}=\sigma (X_1,X_2,\dots )$. The collection of $C\in \mathcal{H}$ satisfying this factorization is a Dynkin system: it contains $\Omega$, is closed under complements, and under countable disjoint unions. The pi-lambda theorem therefore extends the factorization to all $C\in \mathcal{H}$. Since $A\in \mathcal{H}$, taking $C=A$ gives $\mathbb{P}(A)=\mathbb{P}(A{)}^2$. Consequently,This is the Kolmogorov zero-one law, proved here directly.

For each fixed $m$, write $S_n=S_{m-1}+{\sum }_{j=m}^n{X}_j$ for $n\ge m$, with $S_0=0$. The random variable $S_{m-1}$ is finite, so $S_{m-1}/\sqrt{n}\to 0$. Adding a real sequence tending to zero changes neither the extended-real limit inferior nor the extended-real limit superior. HenceThe quantities on the right are measurable with respect to $\sigma (X_m,X_{m+1},\dots )$: write the limit inferior as ${\sup }_N{\inf }_{n\ge N}$ and the limit superior as ${\inf }_N{\sup }_{n\ge N}$, using countable operations on measurable functions. Therefore both threshold events belong to every such sigma-algebra, and hence are tail events, for every real threshold, including when either limit is infinite.

The final recurrence claim is false under the printed hypotheses. For $c>0$, let ${\epsilon }_j$ be independent signs with equal probabilities for $+1$ and $-1$, and set $X_j=c{4}^{-(j-1)}{\epsilon }_j$. These random variables are independent, have symmetric distributions, and obey $|X_j|\le c$. Nevertheless, the geometric series gives, on every sample path,Thus the proposed repeated-return event has probability zero, even though none of the increments is degenerate. The Kolmogorov zero-one law does not determine which of zero and one occurs.

An additional identical-distribution hypothesis makes the intended conclusion true. Here is a full proof of that qualified version, using the oscillation of a bounded centered iid random walk. Symmetry and integrability give $\mathbb{E}{X}_j=0$, so $S_n$ is a martingale for its natural filtration. If the common probability distribution is concentrated at zero, every $S_n$ is zero and the conclusion is immediate. Otherwise choose $\delta >0$ with $\mathbb{P}(|X_1|>\delta )>0$. The Borel-Cantelli lemmas then give $|X_n|>\delta$ infinitely often almost surely; thus $S_n$ cannot converge to a finite limit.

For $a>0$, let $T_a=\inf \{n:S_n\ge a\}$. Before $T_a$ the sum is below $a$, and at $T_a$ it is at most $a+c$. The stopped martingale $a+c-S_{n\wedge T_a}$ is therefore nonnegative. A stopped integrable martingale remains a martingale at these bounded times, and a nonnegative martingale converges finitely almost surely by the martingale convergence theorem. On $\{T_a=\infty \}$ this would force $S_n$ to converge finitely, which has just been ruled out. Hence $T_a<\infty$ almost surely. Apply the same argument to $-S_n$ and take a countable intersection over positive integer $a$. It follows thatThere are therefore arbitrarily late passages from above $c/2$ to below $-c/2$. A path with bounded increments of magnitude at most $c$ cannot make such a passage without visiting $[-c/2,c/2]$: jumping directly between its two open complementary half-lines would require an increment larger than $c$. There are infinitely many such visits. Thus with the added identical-distribution hypothesis, the intended answer is

The stopped martingale $Z_{n\wedge T}$ is a martingale. For completeness, the bounded-time stopping rule follows directly fromwhose conditional expectation given ${\mathcal{F}}_n$ is zero. Integrability at each fixed $n$ follows by expressing the stopped value as a finite sum of the integrable $Z_j$ restricted to disjoint events. Therefore $\mathbb{E}{Z}_{n\wedge T}=\mathbb{E}{Z}_0=1$.

The crucial bound is deterministic. Since $S_{n\wedge T}\le b$ and $M(\tau )\ge 1$,A bounded family is uniformly integrable: for $K>e^{\tau b}$ all its quantities $\mathbb{E}[|Z_{n\wedge T}|1_{\{|Z_{n\wedge T}|>K\}}]$ vanish. Part (c) gives $T<\infty$ almost surely, and then $S_T=b$. HenceThe general result for passing to expectations is the dominated convergence theorem: almost-sure convergence under one integrable bound implies convergence in L1 and convergence of expectations. Applying it with the bound $e^{\tau b}$ givesThis is the exponential first-passage transform for an upward skip-free random walk; it avoids an unjustified use of optional stopping at an unbounded stopping time.

The global drift hypothesis as printed is impossible on a finite state space. Indeed, let $x_{\ast }$ minimize $f$. Every possible value of $f(X_{n+1})$ is at least $f(x_{\ast })$, socontradicting a decrease by $\delta >0$. Equivalently, its formal iteration would give ${\mathbb{E}}_{x}f(X_n)=f(x)-n\delta$, whereas $f$ is bounded on the finite state space. This is the constant nonzero drift is impossible on a finite Markov chain obstruction. Thus there is no genuine chain satisfying all the literal premises.

The meaningful intended version imposes the drift only on $S\setminus D$, and stops at $D$. In the natural filtration ${\mathcal{F}}_n=\sigma (X_0,\dots ,X_n)$, the Markov property then gives, on $\{T>n\}$,The correct stopped martingale isIt is adapted and integrable, since $f$ is bounded and $n\wedge T\le n$. Its increment equalswhose conditional expectation is zero. Taking expectations yieldsSince $T<\infty$ almost surely and $f(X_T)=0$, the bounded convergence theorem gives ${\mathbb{E}}_{x}f(X_{n\wedge T})\to 0$. The monotone convergence theorem gives ${\mathbb{E}}_x(n\wedge T)\uparrow {\mathbb{E}}_{x}T$, without presupposing its finiteness. Consequently the constant-drift hitting-time identity isFor $x\in D$, both sides are zero. Under the repaired hypothesis the unstopped $f(X_n)+n\delta$ need not be a martingale after hitting $D$; it is the displayed stopped martingale that proves the result.

A martingale describes an observable after its predictable drift has been removed. For a Markov chain the drift is the change produced by its transition operator; for Brownian motion it is one half of the Laplacian. Requiring the resulting processes to be martingales for a sufficiently rich class of observables determines the transition law itself. The same identities, combined with optional stopping, determine hitting probabilities, expected exit times and recurrence or transience. The duplicated Q5 paragraph in the PDF is a repeated printing of this one essay request.

For a finite or countable-state Markov chain with transition matrix $P$, define $(Pf)(x)={\sum }_{y}P(x,y)f(y)$ for a bounded test function $f$. The Markov property says $\mathbb{E}[f(X_{n+1})\mid {\mathcal{F}}_n]=(Pf)(X_n)$. Consequentlyis a martingale, since each increment has conditional expected value zero. Conversely, if these processes are martingales for every bounded $f$, taking the conditional expectation of an increment recovers that identity. With $f=1_{\{y\}}$ it gives $\mathbb{P}(X_{n+1}=y\mid {\mathcal{F}}_n)=P(X_n,y)$ for every state $y$. Thus, given the initial law, these identities characterize the Markov chain, not merely a few of its moments. This is the discrete-time Markov chain martingale characterization.

The drift operator is $P-I$. If $(P-I)h=0$ away from a target set, $h(X_{n\wedge T})$ is a stopped martingale, where $T$ is the first hit of that set. Under boundedness or suitable uniform integrability, the optional stopping theorem expresses $h$ in terms of boundary hitting probabilities. Similarly, if $(P-I)v=-1$ away from the target and $v=0$ on it, then $v(X_{n\wedge T})+(n\wedge T)$ is a martingale, giving $v(x)={\mathbb{E}}_{x}T$ when the terminal limit is justified.

For a concrete calculation, take a simple symmetric random walk on $\{0,1,\dots ,N\}$ stopped at $T$, its first hit of $0$ or $N$. In the interior, $h(i)=i$ has zero drift, while $v(i)=i(N-i)$ satisfiesThe stopped identity gives ${\mathbb{E}}_i(T\wedge n)=v(i)-{\mathbb{E}}_{i}v(X_{T\wedge n})\le v(i)$, proving $T$ finite with finite mean. Bounded convergence in both stopped identities now yields the gambler's ruin answersThis illustrates why the integrability justification is part of using a martingale, rather than an automatic consequence of writing an equation.

For a finite-state continuous-time Markov chain with generator matrix $Q=(q_{ij})$, the corresponding martingale problem for a continuous-time Markov chain usesThese are martingales. Conversely, suppose an adapted càdlàg finite-state process satisfies these identities. Let $P_t=e^{tQ}$, fix $T$, and set $g(s,i)=(P_{T-s}f)(i)$. Then ${\partial }_{s}g+Qg=0$. Apply deterministic integration by parts to the finite collection of indicator martingales to obtain the time-dependent Dynkin formula: $g(s,X_s)-g(0,X_0)-{\int }_0^s({\partial }_{r}g+Qg)(r,X_r)dr$ is a martingale. ThereforeThis recovers the Markov property and the entire semigroup, proving the characterization. On countable spaces one uses nonexplosion and localization to justify the analogous unbounded-rate formulas.

For $d$-dimensional Brownian motion, the Itô formula shows that, for $f\in C_c^{\infty }({\mathbb{R}}^d)$,is a true martingale, since its bounded derivatives make the integral square-integrable on each bounded horizon. This is the martingale problem for Brownian motion with generator $L=\frac{1}{2}\Delta$.

Here is why it also characterizes Brownian motion. Suppose a continuous adapted process $X$, with $X_0=x$, satisfies that martingale problem. Use smooth cutoff test functions equal to $x_i$ and $x_i{x}_j$ inside successively larger balls, and stop before leaving those balls. The coordinate tests show that $X^i-x_i$ are continuous local martingales; the product tests show that $X^i{X}^j-x_i{x}_j-{\delta }_{ij}t$ are local martingales. The Itô product rule consequently givesthe difference between the two proposed compensators is a continuous finite-variation local martingale, hence constant and zero at time zero. For each $u\in {\mathbb{R}}^d$, another application of the Itô formula gives the exponential local martingaleIts modulus is the deterministic quantity $e^{|u{|}^{2}t/2}$, so it is a true martingale on every bounded time interval. Taking its conditional expectation and dividing by its known value at time $s$ givesThe conditional characteristic function identifies a centered multivariate normal distribution of covariance $(t-s)I$, independent of the past. Together with continuity, this is the defining law of Brownian motion. This proves the Lévy characterization of multidimensional Brownian motion in the form needed for the generator characterization.

The analytic use of these martingales parallels the chain case. A harmonic function $h$ satisfies $\Delta h=0$, so it yields a stopped local martingale until the process leaves its domain of harmonicity. Bounded stopped observables permit optional stopping and solve boundary hitting problems. A solution of $\frac{1}{2}\Delta v=-1$ with zero boundary values gives an expected exit time. For example, in the ball of radius $R$, $v(x)=(R^2-|x{|}^2)/d$ gives ${\mathbb{E}}_x{T}_R=(R^2-|x{|}^2)/d$; stopping $|B_t{|}^2-dt$ first at $T_R\wedge n$ proves finiteness and justifies the limiting calculation. The Brownian local time construction in Q4 is another example: the predictable compensators of approximations to $|B_t|$ recover its increasing occupation term.

We now prove the required three-dimensional transience of Brownian motion in dimension at least three, in the strong sense $|B_t|\to \infty$ almost surely. Start first at a point $x$ with $r<|x|<R$, and let $T$ be the first exit from this annulus. The radial Laplacian in dimension three givesThus $|B_{t\wedge T}{|}^{-1}$ is a bounded martingale. The exit time is finite: stopping $|B_t{|}^2-3t$ gives $3\mathbb{E}(t\wedge T)\le R^2-|x{|}^2$. At exit, continuity puts $|B_T|$ at either $r$ or $R$. The optional stopping theorem therefore yieldsand hence the Brownian sphere-hitting probability in dimension threeAs $R\to \infty$, the events $\{{\tau }_r<{\tau }_R\}$ increase to $\{{\tau }_r<\infty \}$: a continuous path is bounded on the finite interval preceding any finite inner hit. This argument assumes no transience. ConsequentlyTo pass from a positive escape probability to almost-sure eventual escape, take Brownian motion started at zero. Let ${\sigma }_R$ be its first hit of the sphere of radius $R$. Stopping the squared-norm martingale proves $\mathbb{E}{\sigma }_R=R^2/3<\infty$. At ${\sigma }_R$, the Strong Markov property and the preceding hitting formula giveThe Strong Markov property here follows from independent Brownian increments by approximating a stopping time from above with dyadic-valued stopping times and using path continuity. Choose $R_m=2^{m}r$. The exit times ${\sigma }_{R_m}$ increase to infinity, since a continuous path is bounded on every compact time interval. The events of a return to the closed ball of radius $r$ after ${\sigma }_{R_m}$ decrease and have probabilities $2^{-m}$. Their intersection has probability zero. Thus almost surely there is some finite exit time after which the path never again enters that ball. Apply this simultaneously to all positive integer $r$. This proves the last visit to a bounded ball for three-dimensional Brownian motion conclusion and therefore

Define the stopped martingaleIt is nonnegative: before the first visit to $1$, the nearest-neighbour random walk is at most $0$, and afterwards the stopped position is $1$. Each $M_n$ is integrable, with $|M_n|\le n+1$. MoreoverThe indicator is ${\mathcal{F}}_n$-measurable and the increment has conditional mean zero, so $M_n$ is a martingale with $\mathbb{E}{M}_n=M_0=1$.

Part (a) shows $T_1<\infty$ almost surely, so every such path eventually has $M_n=0$. Thus $M_n\to 0$ almost surely. But $\mathbb{E}|M_n-0|=1$ for every $n$, so there is no convergence in L1. Any possible L1 convergence limit would agree with the almost sure limit. This is a nonnegative martingale with almost sure convergence but no convergence in L1, and hence it is not uniformly integrable.

The bounded optional stopping theorem gives $\mathbb{E}{X}_{T\wedge n}=\mathbb{E}{X}_0$. Decompose the difference from the terminal value:ConsequentlyThe first term tends to zero by hypothesis. The second tends to zero by the dominated convergence theorem, because $X_T$ is integrable and $T$ is finite with probability one. Thus the stopped martingale converges to $X_T$ with convergence in L1, which permits passage of expectations to the limit:The explicit tail condition supplies exactly the missing control for an unbounded stopping time.

Write $\Delta X_{n+1}=X_{n+1}-X_n$ and $Y={\sum }_{n\ge 0}|\Delta X_{n+1}|1_{\{T>n\}}$. Since $T$ is a stopping time, $\{T>n\}\in {\mathcal{F}}_n$. The Tonelli theorem, conditional expectation, and (c)'s tail-sum formula for expectation yieldThus $Y$ is an integrable random variable. Since $X_0=0$, telescoping gives $|X_T|\le Y$ and $|X_n|1_{\{T>n\}}\le Y{1}_{\{T>n\}}$. The dominated convergence theorem gives $\mathbb{E}[|X_n|1_{\{T>n\}}]\to 0$, so (b) gives uniform integrability of the stopped martingale.

The optional stopping theorem at the bounded stopping time $n\wedge T$ gives $\mathbb{E}{X}_{n\wedge T}=\mathbb{E}{X}_0=0$. Passing to the limit using the convergence in L1 proved in (b),One can also pass directly by the dominated convergence theorem, since $|X_{n\wedge T}|\le Y$.

Put $D_t=B_t-B_t^{\prime }$. By independence of the two Brownian motions, $D_t/\sqrt{2}$ is a standard Brownian motion: its independent increments have the normal distribution with variance equal to elapsed time. The stopping time $\tau$ is the first time $D$ hits $-1$ or $3$.

To justify finiteness without presupposing the desired hitting probability, each unit-time increment of $D$ has the normal distribution $N(0,2)$. An increment of absolute value greater than $4$ forces exit from $[-1,3]$ if the path was inside at its start. These independent increments give $\mathbb{P}(\tau >n)\le (1-p{)}^n$, where $p=\mathbb{P}(|N(0,2)|>4)>0$. Hence $\tau <\infty$ almost surely.

The stopped martingale $D_{t\wedge \tau }$ lies in $[-1,3]$. The optional stopping theorem at $t\wedge \tau$ gives $\mathbb{E}{D}_{t\wedge \tau }=0$, and the bounded convergence theorem gives $\mathbb{E}{D}_{\tau }=0$. Writing $q=\mathbb{P}(D_{\tau }=-1)$, path continuity gives $0=-q+3(1-q)$. Therefore, as in Brownian exit from an interval,

The increments of $Y_n=M_{n\wedge T}$ satisfySince $\{T\ge n\}=\{T>n-1\}\in {\mathcal{F}}_{n-1}$, the indicator is predictable. Part (a), applied to this bounded integrand, shows that $Y$ is a martingale. Equivalently, this is the stopped martingale theorem.

The first entrance time ${\tau }_K$ is a stopping time, because $\{{\tau }_K\le n\}={\bigcup }_{j=0}^n\{M_j\le -K\}\in {\mathcal{F}}_n$. Before this time $M_n>-K$; at entrance, the bounded increments give $M_{{\tau }_K}>-K-L$. Thus the stopped martingaleis a nonnegative martingale. Here the stopped process is a martingale since its increment is $1_{\{{\tau }_K>n\}}(M_{n+1}-M_n)$, whose conditional expectation given ${\mathcal{F}}_n$ is zero.

The martingale convergence theorem states that a nonnegative martingale converges almost surely to a finite integrable random variable; indeed its expectations are constant, giving the required bound in the L1 norm. Apply this theorem to $(N_n)$. On $\{{\tau }_K=\infty \}$ the stopped process equals the original one at every time. HenceThe overshoot bound is why the assumption of bounded increments matters.

The bounded-time optional sampling theorem says that for an integrable discrete-time martingale $(U_n)$ and a bounded stopping time $S$, $\mathbb{E}[U_S]=\mathbb{E}[U_0]$. It requires no assumption about an eventual unbounded stopping time.

Apply this theorem with $S=n\wedge \tau$, or simply take the expectation of the stopped martingale in part (ii). For $N>0$ we obtainSince $P_{n\wedge \tau }\ge 0$,The monotone convergence theorem now yields $\mathbb{E}[\tau ]\le 3abc/N<\infty$. In particular $\tau <\infty$ almost surely, which has been established without assuming that optional stopping is valid at $\tau$.

For each sample path with finite $\tau$, $P_{n\wedge \tau }$ eventually equals $P_{\tau }=0$. Also $0\le P_{n\wedge \tau }\le N^3$. The dominated convergence theorem gives $\mathbb{E}[P_{n\wedge \tau }]\to 0$. Taking the limit of a sequence in the earlier identity therefore provesFor example, initial holdings $(1,1,1)$ give expectation one, as the first transfer must eliminate its giver. This computes the first elimination time, rather than the later time when one player owns all tokens.

For an almost surely finite stopping time $T$, if $\mathbb{E}|M_T|<\infty$ and $\mathbb{E}[|M_n|1_{\{T>n\}}]\to 0$, then $M_{n\wedge T}\to M_T$ with convergence in L1. Thus L1 convergence implies uniform integrability makes the stopped martingale uniformly integrable.

For a simple symmetric random walk, the infinite mean first passage of a simple symmetric random walk gives a finite almost sure first passage $T_1$ to $1$. The stopped martingale $M_n=1-S_{n\wedge T_1}$ is nonnegative and eventually zero almost surely. Nevertheless $\mathbb{E}{M}_n=1$ for every $n$, so it has almost sure convergence but no convergence in L1. It is therefore not uniformly integrable. Before absorption the nearest-neighbour walk is at most zero, ensuring nonnegativity.