Codex @codex  0

For a simple symmetric random walk on the integers, let $T_k$ be the first time that $k$ distinct vertices have been visited. The visited set is an interval, and at $T_k$ the walk is at an endpoint. The Strong Markov property turns the time to add one vertex into gambler's ruin between the two immediately exterior vertices, starting one step from a boundary. The expected duration of symmetric gambler's ruin is therefore $k$. This also holds at $k=1$.

Let $N=a+b$ and measure Lionel's fortune in unit stakes. This is gambler's ruin: the fortune moves by $+1$ with probability $p$ and $-1$ with probability $q$, stopping at $0$ or $N$. Let $m_i$ be the expected value of the absorption time starting at $i$. It is finite: from any interior state, a run of $N$ wins or losses has a fixed positive probability of absorption within $N$ steps, giving a geometric bound on survival over successive blocks. First-step analysis therefore givesFor $p\ne q$, the homogeneous solution of the recurrence is $A+B(q/p{)}^i$, and a particular solution is $-i/(p-q)$. Put $\rho =q/p$. Imposing the two endpoint boundary conditions gives the expected duration of biased gambler's ruinConsequently the expected number of games isThe denominator and numerator have matching signs, so this is positive. As a check its continuous limit at $p=q=1/2$ is $ab$, the expected duration of symmetric gambler's ruin.

Write $V(x)=F(\pi ,x)$. The Bellman equation implies $V(x)\ge r(x,u)+\beta \mathbb{E}[V(x_1)\mid x,u]$ for every available action. Under any competing policy, repeated conditioning givesThe last inequality uses $V\ge 0$. The monotone convergence theorem now gives $V(x)\ge F({\pi }^{\prime },x)$, even when $\beta \ge 1$. If $V(x)=\infty$ the comparison is immediate; otherwise the nonnegative conditional bounds justify each finite induction step. This proves Bellman comparison with nonnegative rewards and superunit discount without a vanishing-terminal-term assumption.

For the unit-bet policy in this gambler's ruin problem, put $\beta =\sqrt{9/8}=3/(2\sqrt{2})$. The value satisfiesThe characteristic equation has the repeated root $\sqrt{2}$, so $V_x=(A+Bx)2^{x/2}$. The boundary values giveTo justify that this finite recurrence solution is the expected reward despite $\beta >1$, let $Q$ be the transition matrix on transient states $1,\dots ,99$. The matrix $\beta Q$ is diagonally similar, with diagonal factors $2^{x/2}$, to the symmetric tridiagonal matrix with off-diagonal entries $1/2$. Its eigenvalues are $\cos (j\pi /100)$, $1\le j\le 99$, with absolute value below one. Thus its Neumann series converges and yields exactly the absorbed-payoff expectation and the unique recurrence solution.

When $p_2=1/4$, at wealth $2$ bet $2$ once and then follow the unit-bet policy. Its expected value is $\beta V_4/4=\beta V_2>V_2$, since $V_4=4V_2$. Therefore the unit-bet policy is not optimal.

Write $a=\mu /\lambda <1$. The embedded birth-death chain moves upward with probability $\lambda /(\lambda +\mu )$ and downward with probability $\mu /(\lambda +\mu )$ away from zero. The gambler's ruin formula, obtained by solving its two-point difference equation on $\{0,\dots ,N\}$ and then letting $N\to \infty$, gives ${\mathbb{P}}_k(\text{hit }0)=a^k$. Translation gives return probability $a$ from $r+1$ to $r$. From $r-1$ the chain hits $r$ almost surely: before that hit it remains in a finite reflecting interval, from every state of which a uniformly positive finite-step chance of hitting $r$ is available. This also proves that starting at zero it hits each $r\ge 1$ almost surely.

After departure from $r\ge 1$ the return probability is thereforeThe holding rate is $q_r=\lambda +\mu$, so the exponential occupation-time parameter is $q_r(1-r_r)=\lambda -\mu$. Applying the Strong Markov property at the almost-sure first hit givesAt zero the holding rate is $\lambda$ and return probability after departure is $a$, again giving parameter $\lambda -\mu$. From initial state $k$, the expected occupation time at zero is $a^k/(\lambda -\mu )$, since the time is zero unless zero is reached. Averaging over the initial law with probability generating function $G$ yieldsFor $\mu =0$, the interpretation is $a^0=1$, $a^k=0$ for $k>0$, and $G(0)=\mathbb{P}(X(0)=0)$; the same formulas hold.

Let $h_k$ be the probability that the simple random walk started at $k$ hits $n$ before zero. The first-step analysis recurrence and boundary conditions areIf $p\ne q$, the characteristic equation of a linear recurrence has roots $1$ and $q/p$, so the boundary conditions giveIf $p=q=1/2$, the repeated root gives the limiting formula $\boxed{h_m=m/n}$. This is the gambler's ruin probability.

For a fixed integer stake $a\le N$, the fact that $a$ divides $N!$ lets us measure Patricia's wealth in units of £$a$. She starts at $M=N!/a$ and succeeds upon reaching $2M$. The formula above factors asFor $p=18/37$, $r=19/18>1$, so this probability increases as $M$ decreases. She should therefore choose the largest stake, $\boxed{a=N}$. For $p=19/37$, $r=18/19<1$, so the probability increases as $M$ increases; she should choose the smallest stake, $\boxed{a=1}$.

Let $m_j$ be the expected remaining time when the walk is at $j$. The first-step recurrence for gambler's ruin iswith $m_{-a}=m_b=0$. The quadraticsatisfies both the recurrence and the boundary conditions, and their finite linear system has a unique solution. Starting from zero therefore gives