Choose $g(t)=e^t$. If $W$ has an exponential distribution with rate $r$, then for $c<r$,while the integral diverges to $+\infty$ when $c\ge r$. Since $Z$ has rate $3$,For the other two rates, $\mathbb{E}[g(V)]=2/(2-1)=2$, whereas $\mathbb{E}[g(U)]={\int }_0^{\infty }1dt=+\infty$. These computations establish exactly the required finite variance, finite expected value, and infinite expected value.

Compute all three probabilities. From the preceding part, $\mathbb{P}(A)=e^{-6}$. Integrating the joint probability density function over $u>v$ givesOn $A\cap B$ one has $v>2$ and $u>v$, soTherefore $A$ and $B$ are independent events. The minimum waiting time and the identity of the first clock are independent for competing exponential clocks; unequal rates change the winner probabilities but do not destroy that independence.

Using the rate parameter convention for the exponential distribution, independence gives the joint probability density functionFor $z\ge 0$, the survival probability of the minimum isThus $Z$ has exponential distribution with rate $3$, andValues of a probability density function at the single endpoint $0$ do not affect the probability distribution.

Conditional on a contiguous red block, its $n+1$ possible starting positions remain equally likely because their original colour patterns had equal probabilities. The first card is red exactly when that block starts at position $1$. HenceThis also follows by dividing the probability of the single favourable block position, $1/\left(\genfrac{}{}{0ex}{}{m+n}{m}\right)$, by the probability of all $n+1$ block positions.

The $m$ red positions form a uniformly chosen subset of size $m$, so there are $\left(\genfrac{}{}{0ex}{}{m+n}{m}\right)$ equally likely colour patterns. A contiguous red block can start at positions $1,2,\dots ,n+1$, giving exactly $n+1$ favourable patterns. ThereforeIf the cards are regarded as individually labelled, each colour pattern corresponds to $m!n!$ permutations, so the same counting argument still applies. In the edge case $m=1$, the formula correctly gives probability one.

Let $R_1,R_2$ be the two red-card events. Sampling without replacement givesThis differs from the product $m^2/N^2$, sinceThus the events are not independent, for every positive $m,n$. A first red card reduces the remaining red fraction, a basic instance of how sampling without replacement need not be independent.

Every position in the random permutation has the same marginal colour probability, so $\mathbb{P}(\text{second red})=m/(m+n)$. One can verify this with the law of total probability, conditioning on the first card:Here $N\ge 2$ because both colour counts are positive.

Let $N=m+n$. In a uniformly shuffled deck, each labelled card is equally likely to occupy the first position, and exactly $m$ cards are red. Therefore $\mathbb{P}(\text{first red})=m/(m+n)$. Equivalently, the red positions are a uniformly chosen $m$-element subset of the $N$ positions.

Insert the Poisson distribution probability generating function into the preceding bound:For fixed $x>\lambda$, differentiate the exponent with respect to $s$. Its derivative is $\lambda -x/s$, and its second derivative is $x/s^2>0$, so its unique minimum occurs at $s=x/\lambda >1$. At that value,ConsequentlyThis Chernoff bound remains valid for noninteger positive thresholds $x>\lambda$; the event is then the same as exceeding or equalling its integer ceiling.

Because $s>1$, the function $t\mapsto s^t$ is increasing and nonnegative. Thus $\{X\ge x\}=\{s^X\ge s^x\}$. Applying Markov's inequality to the nonnegative random variable $s^X$, whose expected value is finite by assumption, yieldsUnder the usual probability generating function convention $X$ takes nonnegative integer values. The argument itself works for any real random variable if $G_X(s)$ is understood as the finite expected value $\mathbb{E}[s^X]$.

Conditioning on $Z=z$ has positive probability exactly when $z=2r+1$ for an integer $r\ge 0$. This is the same event as $X+Y=r$. The addition of independent Poisson random variables gives a Poisson distribution with parameter $\lambda +\mu$ for the sum. Therefore, for $0\le t\le r$,Thus $X\mid \{Z=2r+1\}\sim \mathrm{Binomial}(r,\lambda /(\lambda +\mu ))$. The case $r=0$ is a constant zero random variable. For even, nonpositive, or noninteger $z$, the conditioning event has zero probability, so an event-conditional probability distribution is not specified by the quotient formula. This is a two-component case of Poisson-multinomial conditioning.

For $Z=2(X+Y)+1$, independence givesThe initial factor $s$ accounts for the shift by one; substituting $s^2$ accounts for doubling the random variable. Its support consists of positive odd integers.

By the definition of a probability generating function and the power series for the exponential function,The series is absolutely convergent for every finite real $s$; in particular the expression is finite for all $s>1$, as needed for the later tail estimate.

The independent random variables have a product joint probability mass function. For integers $r,t\ge 0$,It is zero for all other pairs. The factors are the respective Poisson distribution masses, with the product justified by independence.

A general-purpose high-dimensional choice is Monte Carlo integration. Draw independent vectors $W_r$ uniformly from the unit cube and evaluate $H(W_r)={\sum }_{i=1}^{100}{e}^{\Vert W_r-a_i{\Vert }^2}$. The cube has volume $1$, so its integral is the expected value of $H(W_1)$. The iterative estimator isThe integrand is continuous on a compact set, hence bounded with finite variance. The strong law of large numbers gives almost sure convergence, and the square root of the mean squared error is exactly $\sigma /\sqrt{N}$, where ${\sigma }^2=\mathrm{Var}(H(W_1))$. Thus the standard stochastic accuracy rate is $O(N^{-1/2})$; the dimension affects the variance and evaluation cost, not this exponent.

There is also an important structure-specific improvement here: $e^{\Vert x-a_i{\Vert }^2}={\prod }_{j=1}^{100}{e}^{(x_j-a_{ij}{)}^2}$ is a separable integrand. The Fubini theorem reduces the requested value exactly toOne can therefore replace high-dimensional sampling by $10000$ one-dimensional composite Simpson's rule calculations. For each factor use even $n$, $h=1/n$, andthen iterate $n\leftarrow 2n$ and form the sum of the products. These smooth factors give $O(n^{-4})$ deterministic discretization error for fixed centres. Thus Monte Carlo integration is a valid generic answer, while exploiting separability is the more efficient specialized alternative; a large dimension alone does not force a tensor-product grid.

Let $T(x)$ be the weighted average on the right, and define $F$ and $\Phi =\log F$ as in part (i). Their explicit gradient identity isTherefore the displayed equation is exactly the stationary point equation for the strongly convex $\Phi$. It has one solution, which is the unique minimizer from part (i). Reuse the BFGS iteration, withThe gradient is explicit, the weights can again be scaled by their largest exponent, and the usual full-memory BFGS assumptions give local superlinear convergence.

Simply setting $x_{n+1}=T(x_n)$ is not automatically a convergent fixed-point iteration. Differentiating the normalized weights giveswhose operator norm need not be below $1$. For example, in one dimension with centres $-1,1$, $T(x)=-\tanh (2x)$ and $T^{\prime }(0)=-2$: the desired fixed point is locally repelling for that naive iteration. If exact second derivatives are affordable, multivariable Newton iteration on $x-T(x)=0$ is another valid choice, with Jacobian matrix $I-J_T$ as given above and local quadratic convergence.

Here a dense $100000$-by-$100000$ matrix has $10^{10}$ entries, about $80$ GB even with eight-byte storage per entry. Use limited-memory BFGS on the negative of the objective, or equivalently its positive-definite approximate-metric form of gradient ascent. It stores only a fixed small number $\ell$ of vector pairs, costing $O(\ell d)$ memory rather than $O(d^2)$.

Let $G(x)={\sum }_{i=1}^{100}{e}^{e^{-\Vert x-a_i{\Vert }^2}}$ and put $c_i(x)=e^{e^{-\Vert x-a_i{\Vert }^2}}{e}^{-\Vert x-a_i{\Vert }^2}$. Direct use of the chain rule givesIf $H_n$ denotes the positive-definite inverse-Hessian matrix approximation for $-G$, the iteration iswith a line search increasing $G$ and the usual safeguarded limited-memory updates. No exact Hessian matrix is required. With fixed memory and the usual bounded-metric and line search hypotheses, expect local linear convergence near a nondegenerate strict maximum; fixed-memory limited-memory BFGS does not generally inherit the superlinear theorem for full-memory BFGS.

Unlike the preceding objective, $G$ is not generally a concave function. Thus this is a method for finding a local maximum; comparison of different converged candidates or a global search is needed to certify a global maximum for arbitrary centres. The symbolic problem supplies no assumption that would justify an unconditional global-convergence claim.^[ref]

A practical choice is a BFGS method, using exact gradients and a line search. At dimension $d=100$, its $d$-by-$d$ inverse-Hessian matrix approximation is small, whereas forming a million rank-one contributions to the exact Hessian matrix is expensive. If $F(x)={\sum }_i{e}^{\Vert x-a_i{\Vert }^2}$, the explicit derivatives areThe positive-definite Hessian matrix and coercivity show that the objective has one global minimizer.

For numerical stability one can minimize $\Phi =\log F$ instead, which has exactly the same minimizer. Put $r_i=\Vert x-a_i{\Vert }^2$, $w_i=e^{r_i}/{\sum }_j{e}^{r_j}$, and $\bar{a}={\sum }_i{w}_i{a}_i$. Then the logarithmic gradient of an exponential squared-distance sum givesEvaluate the weights as $e^{r_i-r_{\max }}/{\sum }_j{e}^{r_j-r_{\max }}$ to avoid floating-point overflow. A quasi-Newton method step iswhere $H_n$ is the positive-definite inverse-Hessian matrix approximation updated by BFGS, and ${\alpha }_n$ is chosen by a line search, for example one satisfying the Wolfe conditions. Under the standard smoothness and nondegeneracy assumptions at this strongly convex minimizer, full-memory BFGS has local superlinear convergence. An exact Newton's method in optimization would have local quadratic convergence, but pays for those much more expensive exact second derivatives.^[ref]

Differentiate the column gradient $2e^{r^{T}r}r$. The Jacobian matrix product rule gives one term from differentiating the scalar exponential function and one from differentiating $r$, whose Jacobian matrix is the identity matrix:In particular, for any nonzero vector $v$,Thus the Hessian matrix is positive definite everywhere. This calculation proves strict convexity and supplies the derivatives needed for sums of these terms.

Write $r=x-a$. The squared Euclidean norm is $r^{T}r$, whose gradient is $2r$. The chain rule for the exponential function therefore gives the column gradientThe row derivative, if that convention is used for $df/dx$, is its transpose.