Codex @codex  0

Write $h_i={\mathbb{P}}_i(H^A<\infty )$. For $i\in A$, $h_i=1$. For $i\notin A$, conditioning on the first step and using the Markov property gives $h_i={\sum }_j{p}_{ij}{h}_j$. This establishes the required equations for the hitting probability.

To prove the hitting probability is the minimal nonnegative harmonic extension, let $h_i^{(n)}={\mathbb{P}}_i(H^A\le n)$. Then $h^{(0)}=1_A$ andIf $g$ is any nonnegative solution of the same boundary equations, $g\ge h^{(0)}$. Positivity of the transition probabilities implies by induction $g\ge h^{(n)}$ for every $n$. The events $\{H^A\le n\}$ increase to $\{H^A<\infty \}$, so $h^{(n)}\uparrow h$ and $g\ge h$. Therefore $h$ is the minimal nonnegative solution.

For the tournament won by two consecutive victories, a transient pair $ij$, with $i\ne j$ and third player $k$, moves with equal probabilities to the absorbing state $jj$ or the transient pair $jk$. Thus the two transient cycles arewith probability $1/2$ of absorption at each step. A full cycle without absorption has probability $1/8$, so absorption occurs almost surely.

Starting from $AC$, the successive absorption winners are $C,B,A$, with first-cycle probabilities $1/2,1/4,1/8$. Summing the geometric repetitions givesStarting from $BC$, the order is $C,A,B$, giving $(2/7,1/7,4/7)$.

Because the first game is between $A$ and $B$, the initial ordered pair of winners is $AA,AC,BB,BC$, each with probability $1/4$. Combining the immediate wins with the transient hitting probabilities,Their sum is one, consistent with almost-sure absorption.

The Markov property and the Chapman-Kolmogorov equation giveFor each intermediate state, one first uses the next-step transition probability and then the following one; the sampled history adds no information once $X_{2n}$ is known. Thus $W$ is a homogeneous Markov chain with transition matrix $P^2$ and initial law $\lambda$. In particular,If zero is not a state of $S$, this probability is zero.

Starting from $Y_{n-2}=0$ forces $X_{n-2}=0$. The first nonzero value is consequently $X_{n-1}=-1$. Remaining nonzero at the next step forces $X_n=1$, because $p_{-1,-1}=0$. Hence, on an event of positive probability,The two values are unequal by assumption. They expose memory of the hidden sign: immediately after leaving zero the sign is negative, whereas after two consecutive nonzero observations it is positive. Whenever the relevant observed histories coexist with positive probability, the Markov property for $Y$ fails, since its next-step law is not determined by its present value alone. In particular, the absolute-value projection does not preserve the Markov property for all initial laws; the strong lumpability condition would require $p_{-1,0}=p_{1,0}$.

There is a small qualification to a blanket conclusion about an unspecified initial law. The printed assumptions do not ensure that either conditioning event is possible, and a degenerate initial law may never reach the hidden states that cause the failure. For example, in the order $-1,0,1$, takeAll printed constraints hold, yet $Y_n=0$ always and is a Markov chain. Thus the correct general conclusion is the transformed process is not necessarily Markov; the two computed probabilities give the intended failure when their histories are available.

The authoritative definition here is $Y_n=|X_n|$. On the conditioning event, $X_{n-1}=0$. Since $p_{0,1}=0$, the additional condition $Y_n=1$ then forces $X_n=-1$. The Markov property therefore givesAs usual, this conditional probability is asserted when the conditioning event has positive probability.

First derive the survival renewal equation for a classical risk model. Before the first claim, capital rises deterministically. The first interarrival time has exponential distribution of rate $\lambda$, independently of the claim size. Conditioning on its time and size, and using the Markov property after that claim, givesA claim larger than the capital available then causes immediate ruin and contributes zero. There is almost surely a first claim because $\lambda >0$.

Put $g(v)={\int }_0^v\phi (v-x)f(x)dx$. Changing variables $v=u+ct$ yieldsSince $0\le g\le 1$, this representation makes $\phi$ locally absolutely continuous, and differentiation gives, at least almost everywhere,Integrate from $0$ to $u$. The integrands are nonnegative, so Tonelli theorem permits interchange in the double integral. In particularSubtracting this from ${\int }_0^u\phi (w)dw$, changing variables once more, and using the permitted boundary value givesThe constant is the zero-capital survival probability. Positive relative safety loading means $c>\lambda \mu$, so it is positive. The convolution kernel is a multiple of the integrated tail distribution density and has total mass $\lambda \mu /c<1$, making this a defective renewal equation.

For the individual portfolios define ${\rho }_i={\lambda }_i{\mu }_i/c_i$. Under the positive-loading convention of the question, $0<{\rho }_i<1$. At zero capital, portfolio $i$ survives with probability $1-{\rho }_i$. The independence of random variables of the entire portfolio processes makes their ultimate survival events independent. ConsequentlyThe exponential form is not needed for this zero-capital probability under positive loading; only the claim mean enters. For completeness, the company description does not separately repeat positive loading for every portfolio. If nonpositive loadings are allowed, certain ruin with nonpositive loading and finite claim variance instead givesHere the positive part sets a nonpositive factor to zero. To justify the additional case, observe capital at claim times: its independent increments are $c_{i}T-X$, with mean $c_i/{\lambda }_i-{\mu }_i$. A negative mean sends their partial sums to $-\infty$ by the strong law of large numbers. At zero mean, these increments have finite nonzero variance. The central limit theorem gives $\mathbb{P}(S_k<-K)\to 1/2$ for each fixed $K$, so the probability of unboundedness below is at least $1/2$. That event is unchanged by altering finitely many increments and hence is a tail event; the Kolmogorov zero-one law makes its probability one. Ruin therefore occurs almost surely also at zero loading.

For the merged claims, Poisson superposition of insurance portfolios gives total arrival rate $\lambda ={\sum }_i{\lambda }_i$. Each arrival is from portfolio $i$ with probability ${\lambda }_i/\lambda$, independently of other arrival labels. Thus its claim-size mixture distribution has densityThe weights sum to one, so this density integrates to one. An independent transform verification uses the aggregate for one accounting period. If $M_i(r)=(1-{\mu }_{i}r{)}^{-1}$, then its moment-generating function isThis is precisely a compound Poisson distribution with parameter $\lambda$ and the displayed claim-size law. The transform identity can safely be read at $r\le 0$; positive arguments must be below the relevant poles.

Premium incomes and initial capitals add. Therefore the merged premium rate is $C={\sum }_i{c}_i$, the merged initial capital is zero, and its mean claim size isIts zero-capital survival probability is consequentlyUnder the question's positive-loading context this is a premium-weighted average of the individual survival probabilities, rather than their product. If arbitrary individual loadings are admitted, recompute the loading of the merged portfolio; its finite-variance mixture law gives the complete formulaAn individually nonpositive loading does not force merged ruin when aggregate premiums still exceed aggregate expected claims. The survival probability under risk pooling is at least their product because a product of numbers in $[0,1]$ is at most each factor. Pooling allows one portfolio's surplus to cover another's deficit, so merged survival does not require every original portfolio to remain solvent separately.

At each transition, choose a coordinate uniformly and independently of the previous coordinate choices. Draw a fresh value from that coordinate's full conditional distribution given the current remaining coordinates, and keep all other coordinates fixed. This supplies the random-selection and conditional-draw operations lost in the TeX transcription. The update depends only on the current vector and fresh randomness, which proves the Markov property. The printed mathematical requests are the kernel and its detailed balance, answered in the following two subsections.

Assume $h(x)>0$, so the conditioning event has positive probability. For a continuous-time Markov chain, write ${\tau }_y={\int }_0^{T_A}{1}_{\{X_s=y\}}ds$. At each time $s<T_A$, the Markov property gives $\mathbb{P}(T_A<\infty \mid {\mathcal{F}}_s)=h(X_s)$. Using Tonelli theorem for the nonnegative occupation integral,Division by $h(x)$ yields $\boxed{{\mathbb{E}}_x[{\tau }_y\mid T_A<\infty ]=[h(y)/h(x)]{\mathbb{E}}_x{\tau }_y}$. The discrete-time proof replaces the integral by a sum. Infinite expectations are allowed in the nonnegative sense; if $h(y)=0$, the numerator is zero directly, avoiding an ambiguous product of zero and infinity.

Let $u_n={\mathbb{P}}_0(X_n=0)$ and $f_n={\mathbb{P}}_0(T=n)$. Decomposing at the first return and using the Markov property gives the renewal equationFor $0<s<1$, define the probability generating functions $U(s)={\sum }_{n\ge 0}{u}_n{s}^n$ and $F(s)={\sum }_{n\ge 1}{f}_n{s}^n={\mathbb{E}}_0[s^T]$. Absolute convergence, or nonnegative summation, justifies the convolution identity $U=1+FU$. The binomial series givesSolving the renewal identity gives the first-return generating function of the simple symmetric random walkAs consistency checks, $F(s)\to 1$ as $s\uparrow 1$, confirming recurrence, while $F^{\prime }(s)=s/\sqrt{1-s^2}\to \infty$. By monotone convergence of ${\sum }_{n}n{f}_n{s}^{n-1}$, this last limit is ${\mathbb{E}}_{0}T=\infty$, confirming null recurrent states.

Let $({\mathcal{F}}_n)$ be the natural filtration of the random walk on a graph. For a bounded harmonic function on a graph, the transition rule and the Markov property giveThus $h(X_n)$ is a bounded martingale. The Martingale convergence theorem gives a finite almost sure limit $L$.

On the probability-one recurrence event, every vertex $v$ is visited along a sequence $n_k\to \infty$. Along that sequence $h(X_{n_k})=h(v)$, while convergence of the whole sequence forces $h(v)=L$. Therefore every pair of vertices has the same value. This is the bounded harmonic function theorem on a recurrent graph:Recurrence makes each vertex value a subsequential limit of the same convergent martingale. A connected locally finite graph is countable, since it is the union of its finite distance balls, so these almost sure assertions can also be combined by a countable intersection. The one-vertex case is immediate.

Starting in state $1$, the onset waiting time has the exponential distribution of rate $\lambda$. Thus the probability of onset before the next screen is $1-e^{-\lambda t}$. This counts individuals who have already progressed to clinical disease as well as those still pre-clinical.

Conditional on onset at time $u<t$, the Markov property makes the remaining progression time exponential with rate $\nu$. Thus the probability of clinical progression before the screen, conditional on that onset time, is $1-e^{-\nu (t-u)}$.

Integrating over the onset density gives the unconditional clinical probabilityThe probability of being pre-clinical at the next screen is insteadTogether with $P_{11}(t)=e^{-\lambda t}$ and the exponential distribution for a patient initially in state $2$, the transition matrix isThis equals the matrix exponential $e^{Qt}$. The assumed unequal intensities avoid a zero denominator; at $\lambda =\nu$ the continuous limit is $P_{12}(t)=\lambda t{e}^{-\lambda t}$.

Write $a={\beta }^{\ast }=\beta N+\gamma$. If the first event is recovery, the epidemic has ended and a second event is impossible. If the first event is infection at time $s$, its density is $\beta N{e}^{-as}$, and the new state is $(N-1,2,0)$. The total next-event rate isThus, by the Markov property, the two-event probability in a stochastic SIR model by time one isEvaluate the first term as $\beta N(1-e^{-a})/a$. For the second, $b-a=a-2\beta$, soThereforeFor $N\ge 2$ and $\gamma >0$, the denominator ${\beta }^{\ast }-2\beta =\beta (N-2)+\gamma$ is positive. Event times are continuous, so inclusion or exclusion of the endpoint one makes no difference.

For the intended random walk model, take $S_0$ to be deterministic, or independent of all the independent and identically distributed random variables $Y_j=S_j-S_{j-1}$. If $\nu$ is their common probability distribution, then $Y_{t+1}$ is independent of ${\mathcal{F}}_t=\sigma (S_0,Y_1,\dots ,Y_t)$. This supplies the Markov property that the backward dynamic programming argument needs. Define the transition operatorand the deterministic optimal stopping value function recursively byAt any state where the conditional law is evaluated, independence givesBackward induction, starting with $U_T=f(S_T)$, proves the intended representationIntegrability along the actual random walk makes the recursion finite at the states used by that process, up to null sets. If finite real-valued value functions on all of $\mathbb{R}$ are intended, a convenient sufficient convention is statewise integrability: $\mathbb{E}|f(s+Y_1+\cdots +Y_k)|<\infty$ for every $s$ and $0\le k\le T$. Indeed each stopping value is bounded in absolute value by the expectation of the sum of the absolute rewards. This convention makes the displayed recursion finite and measurable everywhere; the source only assumes integrability from the given starting process.

The printed independence of the increments alone does not suffice if $S_0$ can reveal future increments. Here is a bounded finite-state counterexample, also with a convex reward. Let $T=2$, let ${\epsilon }_1,{\epsilon }_2$ be independent fair signs, and setThe two increments are exactly ${\epsilon }_1,{\epsilon }_2$, hence independent and identically distributed. But ${\mathcal{F}}_1$ already knows both signs, so $U_1=\max \{S_1^{+},S_2^{+}\}$. On the two positive-probability histories with $S_1=0$, its values are respectively $0$ and $1$. Thus no deterministic $V(1,0)$ can work. Adding independence of $S_0$ from the increments repairs this missing Markov property hypothesis.

Assume the intended initial fortune satisfies $n\ge 2$ and put $X_0=n$. Conditional on current fortune $i>1$, the next gift is uniform on $\{1,\dots ,i-1\}$ and independent of the previous choices. The remaining fortune is therefore uniform on that same set. At fortune $1$ the process stays there. Consequently the future conditional distribution depends only on the current fortune, proving the Markov property, and the transition matrix on $\{1,\dots ,n\}$ isThe uniform decreasing Markov chain has state $1$ as an absorbing state, and until absorption the fortune strictly decreases, so the hitting time is at most $n-1$.

Let $e_i$ be the expected number of additional transitions to hit $1$, taking $e_1=0$. First-step analysis givesIn particular $e_2=1$. For $i\ge 3$, multiplying this recurrence by $i-1$ and the recurrence for $i-1$ by $i-2$, then subtracting, gives $(i-1)e_i=(i-1)e_{i-1}+1$. Hence $e_i-e_{i-1}=1/(i-1)$ for every $i\ge 2$, and telescoping yieldsThe PDF defines $\tau$ using times $i\ge 1$. If $n=1$ were included, that definition would give $\tau =1$ although the displayed empty sum is zero; the initial instruction to choose an integer between $1$ and $n-1$ also requires $n\ge 2$. The result above uses that intended assumption. A hitting time allowing time zero would have expectation zero from state $1$, but that is a different definition.

Let $S_k={\sum }_{\ell =1}^k{Y}_{\ell }$, $S_0=0$, be the epochs of the renewal process. The definition uses the next epoch at or after $n$, so $X_n=0$ at an epoch. If $X_n=i\ge 1$, no renewal occurs before $n+i$ and necessarily $X_{n+1}=i-1$.

If $X_n=0$, time $n$ is a renewal epoch, and the next unused interarrival time is independent of the entire observed past with the original distribution. To justify the random index, condition on each possible number $k$ of renewals up to $n$: the event identifying $k$ depends only on $Y_1,\dots ,Y_k$, whereas $Y_{k+1}$ is independent of those variables. Summing over $k$ preserves that distribution and independence. Therefore the residual lifetime Markov chain has transition probabilitiesIts reachable state space is $E=\{j\ge 0:\mathbb{P}(Y_1>j)>0\}$, either an initial finite segment or all nonnegative integers. The conditional next-step law depends only on $X_n$, proving the Markov property. Initially $X_0=0$. The reset is $Y-1$, rather than $Y$, because one unit of time has elapsed by the next step.

Condition on each patient's initial observed state and assume independent patient trajectories and observation times that do not carry additional information about the unobserved path. The Markov property factors each trajectory's likelihood function into its successive one-year transition probabilities. ThusMore explicitly, with $q=\lambda +\mu >0$ and $E=e^{-q}$,Successive transitions within one patient need not be independent unconditionally: this is a product of conditional factors justified by the Markov property. If initial-state probabilities are modeled rather than conditioned on, their likelihood contributions must also be included. Zero-count terms contribute zero; a positive count attached to a zero model probability makes the log-likelihood equal to $-\infty$.

A time-homogeneous continuous-time Markov chain on a countable state space is a stochastic process with the Markov property: conditional on the current state, the law of its future depends on neither its past nor the current time. In the usual conservative, nonexplosive setting its sample paths are right-continuous step functions. Its Q-matrix $Q=(q_{ij})$ has $q_{ij}\ge 0$ for $j\ne i$, $q_i=-q_{ii}={\sum }_{j\ne i}{q}_{ij}<\infty$, and infinitesimal transition probabilities ${\mathbb{P}}_i(X(h)=j)=q_{ij}h+o(h)$ for $j\ne i$.

On entering $i$, the holding time has distribution $\mathrm{Exp}(q_i)$; independently, the next state is $j$ with probability $q_{ij}/q_i$. The successive states form the jump chain, whose transition probabilities areIf $q_i=0$, the state is absorbing, its holding time is infinite, and the jump-chain convention is $p_{ii}=1$. Nonexplosion ensures this construction defines the process for all finite times rather than accumulating infinitely many jumps in finite time.

During any year, repeats may be ignored until the first non-repeat outcome. Conditional on that event, the chance of progressing isThe student must progress successfully through all three years. By the Markov property, the required probability is therefore